Have you ever ever puked and had diarrhea on the similar time? I’ve, and when it occurred, I used to be listening to a fan-made audiobook model of Harry Potter and the Methods of Rationality (HPMOR), a fan fiction written by Eliezer Yudkowsky.
No, the dual-ended bodily horror was not incited by the fanfic, however the two experiences are inextricable in my thoughts. I used to be shocked to find years later that the 660,000-word fanfic I marathoned whereas sick has some weird intersections with the extremely rich technorati, together with lots of the figures concerned within the present OpenAI debacle.
Living proof: In an Easter egg spotted by 404 Media (which was too minor for anybody else — even me, somebody who’s really learn the thousand-odd web page fanfic — to note), there’s a once-mentioned Quidditch participant within the sprawling story named Emmett Shear. Sure, the identical Emmett Shear who co-founded Twitch and was simply named interim CEO of OpenAI, arguably probably the most influential firm of the 2020s. Shear was a fan of Yudkowsky’s work, following the serialized story because it was printed on-line. So, as a birthday current, he was gifted a cameo.
Shear is a longtime fan of the writings of Yudkowsky, as are lots of the AI business’s key gamers. However this Harry Potter fanfic is Yudkowsky’s hottest work.
HPMOR is an alternate universe rewriting of the Harry Potter sequence, which begins with the premise that Harry’s aunt Petunia married an Oxford biochemistry professor as a substitute of the abusive dolt Vernon Dursley. So, Harry grows up as a know-it-all child obsessive about rationalist considering, an ideology which prizes experimental, scientific considering to unravel issues, eschewing emotion, faith or different imprecise measures. It’s not three pages into the story earlier than Harry quotes the Feynman Lectures on Physics to attempt to clear up a disagreement between his adoptive dad and mom over whether or not or not magic is actual. If you happen to thought precise Harry Potter might be somewhat irritating at instances (why doesn’t he ever ask Dumbledore the obvious questions?), prepare for this Harry Potter, who may give the eponymous “Younger Sheldon” a run for his cash.
It is sensible that Yudkowsky runs in the identical circles as lots of the most influential individuals in AI immediately, since he himself is a longtime AI researcher. In a 2011 New Yorker function on the techno-libertarians of Silicon Valley, George Packer stories from a cocktail party on the residence of billionaire enterprise capitalist Peter Thiel, who would later co-found and put money into OpenAI. As “blondes in black attire” pour the lads wine, Packer dines with PayPal co-founders like David Sacks and Luke Nosek. Additionally on the social gathering is Patri Friedman, a former Google engineer who received funding from Thiel to start out a nonprofit that goals to construct floating, anarchist sea civilizations impressed by the Burning Man competition (after 15 years, the group doesn’t appear to have made a lot progress). After which there’s Yudkowsky.
To additional join the events concerned, behold: a 10-month-old selfie of now-ousted OpenAI CEO Sam Altman, Grimes and Yudkowsky.
Yudkowsky will not be a family title like Altman or Elon Musk. However he tends to crop up repeatedly within the tales behind firms like OpenAI, and even behind the good romance that introduced us youngsters named X Æ A-Xii, Exa Darkish Sideræl and Techno Mechanicus. No, actually — Musk as soon as needed to tweet a joke about “Roko’s Basilisk,” a thought experiment about synthetic intelligence that originated on LessWrong, Yudkowsky’s weblog and group discussion board. However, because it turned out, Grimes had already made the identical joke a few “Rococo Basilisk” within the music video for her tune “Flesh With out Blood.”
HPMOR is sort of actually a recruitment software for the rationalist motion, which finds its digital residence on Yudkowsky’s LessWrong. By an admittedly entertaining story, Yudkowsky makes use of the acquainted world of Harry Potter for instance rationalist ideology, displaying how Harry works in opposition to his cognitive biases to change into a grasp problem-solver. In a closing showdown between Harry and Professor Quirrell — his mentor in rationalism who seems to be evil — Yudkowsky broke the fourth wall and gave his readers a “final exam.” As a group, readers needed to submit rationalist theories explaining how Harry may get himself out of a deadly predicament. Fortunately, for the sake of comfortable endings, the group handed.
However the ethical of HPMOR isn’t simply to be a greater rationalist, or as “much less flawed” as you will be.
“To me, a lot of HPMOR is about how rationality could make you extremely efficient, however extremely efficient can nonetheless be extremely evil,” my solely different pal who has learn HPMOR informed me. “I really feel like the entire level of HPMOR is that rationality is irrelevant on the finish of the day in case your alignment is to evil.”
However, in fact, we are able to’t all agree on one definition of excellent versus evil. This brings us again to the upheavals at OpenAI, an organization that’s attempting to construct an AI that’s smarter than people. OpenAI needs to align this synthetic normal intelligence (AGI) with human values (such because the human worth of not being killed in an apocalyptic, AI-induced occasion), but it surely simply so occurs that this “alignment research” is Yudkowsky’s specialty.
In March, 1000’s of notable figures in AI signed an open letter arguing for all “AI labs to instantly pause for at the very least 6 months.”
Signatories included Meta and Google engineers, founders of Skype, Getty Photographs and Pinterest, Stability AI founder Emad Mostaque, Steve Wozniak and even Elon Musk, a co-founder of OpenAI who stepped down in 2018. However Yudkowsky didn’t signal the letter, and as a substitute, penned an op-ed in TIME Magazine to argue {that a} six-month pause isn’t radical sufficient.
“If any person builds a too-powerful AI, below current situations, I anticipate that each single member of the human species and all organic life on Earth dies shortly thereafter,” Yudkowsky wrote. “There’s no proposed plan for the way we may do any such factor and survive. OpenAI’s brazenly declared intention is to make some future AI do our AI alignment homework. Simply listening to that that is the plan must be sufficient to get any smart individual to panic. The opposite main AI lab, DeepMind, has no plan in any respect.”
Whereas Yudkowsky argues for the doomerist method in the case of AI, the OpenAI management kerfuffle has highlighted the wide selection of various beliefs round methods to navigate know-how that’s presumably an existential risk.
Performing because the interim CEO of OpenAI, Shear — now one of the crucial {powerful} individuals on the earth, and never a Quidditch seeker in a fanfic — is posting memes in regards to the totally different factions within the AI debate.
There’s the techno-optimists, who help the expansion of tech in any respect prices, as a result of they assume any issues attributable to this “develop in any respect prices” mentality can be solved by tech itself. Then there’s the efficient accelerationists (e/acc) which appears to be sort of like techno-optimism, however with extra language about how progress in any respect prices is the one means ahead due to the second legislation of thermodynamics says so. The safetyists (or “decels”) help the expansion of know-how, however solely in a means that’s regulated and protected (in the meantime, in his “Techno-Optimist Manifesto,” enterprise capitalist Marc Andreessen decries “belief and security” and “tech ethics” as his enemy). After which there are the doomers, who assume that when AI outsmarts us, it’s going to kill us all.
Yudkowsky is a frontrunner among the many doomers, and he’s additionally somebody who has spent the final many years operating in the identical circles as what looks like half of the board of OpenAI. One common principle about Altman’s ousting is that the board needed to nominate somebody who aligned extra intently with its “decel” values. So, enter Shear, who we all know is impressed by Yudkowsky and likewise considers himself a doomer-slash-safetyist.
We nonetheless don’t know what’s happening at OpenAI, and the story appears to alter about as soon as each 10 seconds. For now, techy circles on social media proceed to combat over decel versus e/acc ideology, utilizing the backdrop of the OpenAI chaos to make their arguments. And within the midst of all of it, I can’t assist however discover it fascinating that, in case you squint at it, all of this traces again to 1 actually tedious Harry Potter fanfic.