SUSPICIOUS MINDS

[00:00:00] SEAN KING O'GRADY: We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far. The sciences, each straining in its own direction, have hitherto harmed us little. But someday, the piecing together of disassociated knowledge will open up such terrifying vistas of reality and of our frightful position therein that we shall either go mad from the revelation or free from the light into the peace and safety of a new dark age.

[00:00:32] HP Lovecraft wrote these words exactly one hundred years ago in a short story called The Call of Cthulhu. This influential text introduced the mythos of Cthulhu into popular culture. This ancient cosmic creature is a giant octopus-faced sea monster, the true description of which defies the meager capabilities of language.

[00:00:55] Even the word Cthulhu is confusing, a jumble of consonants intentionally hard to [00:01:00] pronounce. The passage I just read exemplifies what's known as the Lovecraftian theme. There are truths in this universe too horrible in content or scope for humans to confront. They will drive us mad or usher us back into the ignorant comfort of a new dark age, as Lovecraft imagined.

[00:01:18] In January twenty twenty-six, Dario Amodei, former OpenAI executive and co-founder and CEO of Anthropic, creator of the popular large language model Claude, published a blog on his website, which is darioamodei.com, titled The Adolescence of Technology. In Amodei's blog, he describes some of the very good and very bad things that might happen as a result of AI.

[00:01:42] And this is actually not an unusual claim from the AI glitterati. Something either very good or very bad is going to happen, so we better let them handle this and give them fictional-sounding amounts of money so they can achieve the very good thing, or at the least prevent the very bad thing. One of the sections [00:02:00] is titled Black Seas of Infinity, a reference to the HP Lovecraft passage we opened with about Cthulhu and the dangers of knowledge.

[00:02:08] The Cthulhu mythos is a popular avatar in some Silicon Valley circles. It's dark, mystic, and centers the bold academics who confront Cthulhu as chosen architects of humanity's future. There's a long tradition of this kind of thinking, the reckless pursuit of knowledge and progress and its inevitable consequences, the limits of what human beings should know, the existential dangers of too much knowledge, as paradoxical as that might seem.

[00:02:34] It's in the stories we all know: Prometheus, Pandora, the Garden of Eden. Science now reigns ascendant as the primary mode of intellectual progress. So it is unsurprising that in recent centuries, the nature of these supposedly dangerous pursuits has been technological. I spoke to Dorian Lynskey, journalist and author of the incredibly comprehensive text on apocalypse and culture, Everything Must Go, about the [00:03:00] relationship between dark speculative fiction and this current crop of tech elites.

[00:03:05] DORIAN LYNSKEY: There's such an odd dance there really between the fiction and the science, and that people who work in AI, most of these people working in AI, they've read the stories. There's literally a novel about a computer, it's called Colossus, and it's about a computer that teams up with a-- this American defense computer, and it teams up with a Russian defense computer, and they run the world.

[00:03:27] So it's kind of like Skynet before Skynet. And you've got a scientist in this novel complaining about the Frankenstein trope. They say, "Oh God, people are always comparing what we do to Frankenstein." And it's just such a kind of like Russian doll of sort of fiction and science.

[00:03:45] SEAN KING O'GRADY: When Dorian references Colossus, he does so because Elon Musk has actually named one of his data centers Colossus after the titular sci-fi supercomputer in a nineteen sixty-six book set interestingly in the nineteen nineties that threatens our [00:04:00] world with nuclear obliteration.

[00:04:01] It's tempting to understand this as run-of-the-mill edge lord stuff, naming your data centers after fictional agents of doom. It's actually something I might do if I started a video game company or something. But here's the thing, some of these people might actually believe what they're doing is potentially going to end the world.

[00:04:18] I actually can't think of another example of something like this happening in recent history. Why are so many of the people spearheading AI development so cavalier with a technology that they themselves concede and often even brag has the destructive potential to end the human race?

[00:04:35] DORIAN LYNSKEY: I found this amazing quote from John von Neumann when he was working on the Manhattan Project, and he said to his wife, particular point in the process, he goes, "I think what we're creating might destroy the whole planet."

[00:04:49] He says, "But you know, you've got to follow the science." He was just sort of... Intellectually, he was just like, "Well, what, you know, what are you gonna do? You've got to follow the knowledge." [00:05:00] So I don't think it's that unusual. I think what's happened now is that it becomes a form of marketing. People want-- If you say, my invention that I'm trying to raise capital for and turn into a profitable business is so powerful and world-changing that it can either take us to heaven or hell then that's good marketing

[00:05:24] SEAN KING O'GRADY: And here's Malo Borgon, CEO of the Machine Intelligence Research Institute, better known as MIRI, with a compelling counterpoint to Dorian's, and many people's, cynicism about the AI industry and its leaders' claims

[00:05:38] MALO BOURGON: I've definitely heard this line before that the talk of the risks and catastrophe or human extinction is all kind of just hype to, to talk up their tech more and make it sound more important and powerful to get investment.

[00:05:50] I think that is born out of some sort of default cynical angle that people have, or at least a lot of people have, to tech companies, and they're kinda just mapping it over to the situation. I [00:06:00] think if they kind of like thought it through a little bit more carefully, there's a lot of different ways of hyping your AGI super intelligence project that doesn't need you to talk about human extinction that would be just as effective.

[00:06:12] And I don't think telling people that, you know, you think your technology might literally cause, you know, the end of humanity does actually help with your fundraising. Like, what other company or industry ever in the world, uh, thought it was a good strategy to try and, you know, attract investment to talk about their, you know, technology potentially risking all of human extinction?

[00:06:32] You could talk about the powerful capabilities that it would have. You could talk about how it could, you know, cure cancer. You could talk about how it will empower our government to be a leader on the world stage. I just think it's kind of all crazy and bullshit, uh, that this, this argument that like they're just talking about it, the extinction stuff, as hype.

[00:06:50] Also, I think it's worth noting, basically every single CEO, certainly the three frontier AI companies, Google DeepMind, Anthropic, and OpenAI, [00:07:00] they were all talking about this stuff before they were running these billion-dollar companies working on this stuff. Sam Altman was talking about this stuff in blog posts before, you know, he was even the CEO of OpenAI or right at the start.

[00:07:13] Same thing with Dario. Demis kind of started DeepMind from the start with the premise that AGI was possible and that super intelligence was possible, and that if we get this right, it would be really great, but also with an appreciation of the risks. Like, this isn't a new thing for these people. And so I think it's, like reasonable to be cynical of their motives, and I certainly think, you know, the profit incentives, the potential power that these things can confer can be like distorting incentives that should make us somewhat cynical of the things they say.

[00:07:41] But I think there's just an enormous amount of evidence that we should just take them at face value, which is it makes no sense to say this unless you believe it because it's not good marketing. We have a very large list of academics who work in the AI field. Two of the three godfathers of deep learning, the, the AI paradigm this is all built on, [00:08:00] have double-digit probabilities that they think that this will potentially lead to extinction.

[00:08:04] One of them, Y-Yoshua Bengio, is the most cited scientist of all time. The other one is a Nobel Prize winner, so they don't have any sort of incentive to try and hype up these companies. I mean, Geoffrey Hinton, the Nobel Prize winner, the other godfather we talked about here, left Google so that he could, you know, feel like he had more freedom to talk about this stuff.

[00:08:21] So yeah. I think they just believe it, and they've been thinking about it for a long time.

[00:08:26] SEAN KING O'GRADY: I have so, so, so many questions. How do we evaluate claims about AI and the end of the world? How do we evaluate claims about AI utopia? Is there validity to these apocalyptic assertions or these visions of bounty, or is it all just hype?

[00:08:41] Do the people building this technology even believe this themselves? How does the specter of the end of the world impact our minds? Or have we always thought this way? How do we discuss the technological end times? In the first season of Suspicious Minds, we explored the emerging trends of AI psychosis and chatbot-derived [00:09:00] delusions.

[00:09:00] In this season, we're widening our aperture to examine AI's relationship to a broader phenomenon, the end of the world. The tech industry predicts that artificial intelligence will inevitably usher in a new era of human flourishing, or it might cause the end of the world. Depends on who you talk to. Why is the word apocalypse suddenly on the tip of every tongue?

[00:09:27] Is this a serious concern? Haven't we always been predicting doomsday and always been wrong? Is this historical moment unique? What does the end of the world even mean? I've spent two years speaking with philosophers, scientists, historians, and theologians about the apocalypse and how thinking about it shapes the human mind, and the people actually building the technology that might make our nightmares a reality or our salvation.

[00:09:54] Welcome to season two of Suspicious Minds: AI and the [00:10:00] Apocalypse.

[00:10:04] Doctors Joel and Ian Gold, brothers and authors of the book Suspicious Minds: How Culture Shapes Madness, return to ground our discussions in current psychological and philosophical discourse. They're also here to make sure I don't make things up and get the fundamentals of psychology wrong, though I can't promise I'll always get it right, even with their sage guidance.

[00:10:25] Dr. Joel Gold is mainly a clinical psychiatrist, but he has, like, nine other really impressive jobs, and Dr. Ian Gold is the chair of philosophy at McGill, a brilliant thinker, and one of the world's nicest people. So the first time that I interviewed you for this show was less than a year ago, and I almost find that hard to believe because so much has changed in, in the kind of- Yeah

[00:10:48] cultural conversation around artificial intelligence has changed. From your perspective, how different are things now in terms of culture in [00:11:00] general and our, our relationship to artificial intelligence now than 10 months ago?

[00:11:04] IAN GOLD, PHD: Yeah. I, uh, uh, I think that, um, the effect of large language models on cultural perceptions of AI has been extraordinary.

[00:11:13] I think we can't underestimate it. You have the feeling, I think it's an illusory feeling, but anyway, you have the feeling that you're talking to, uh, something that can be called intelligent, and in fact, I think we can say more strongly, you have the feeling that you're talking to a person. Now, so, so there's a cultural change that comes from having that experience.

[00:11:36] Now, that's a separate question from the question of whether AI is actually progressing in ways that should make us worried or should make us more excited.

[00:11:46] DR JOEL GOLD: Nothing has evolved in my mind this quickly. I mean, I guess going back to the pandemic, you know, things happened in the space of days and weeks, but it wasn't like my thoughts changed.

[00:11:58] But in no [00:12:00] way have, uh, have my thoughts about something changed this dramatically. If we have another conversation in eight months, assuming we're both here, I don't know that it'll change with respect to the, the fear. The kind of the specifics may, may change, but the overall gloom and doom using the Cold War as a parallel and people who were frightened, um, say after the Cuban Missile Crisis were still really scared 10 years later and 20 years later, and to some degree still, although that's been eclipsed, um, largely by, you know, by the fears around AI.

[00:12:42] SEAN KING O'GRADY: Is, is there the possibility that a lot of this apocalyptic thinking we're, we're, are, a l- so many of us are experiencing surrounding AI could be veering into the territory of delusions? [00:13:00]

[00:13:00] IAN GOLD, PHD: That's a hard question. The apocalypse plays a central role in probably the most famous extended study of delusions ever carried out, which is the memoir of, uh, Daniel Paul Schreber called M-Memoir of My, uh, Nervous Illness That's the English translation.

[00:13:22] Schreber was himself German, and the original was written in German. Schreber was a judge at the end of the 19th century who developed psychotic illness, and he spent most of his life in an asylum. And he wrote this memoir of his religious revelations. The central delusion that he had was that God had destroyed the entire world with the exception of him, and that God was planning to repopulate the world with him.

[00:13:57] And in order to repopulate the world with him, he had to become [00:14:00] a woman. And so God was turning him into a woman. The apocalypse has this, instead of this paranoid flavor, right, this frightening flavor, it has a grandiose flavor. Because, of course, the apocalypse has suddenly made him the most important human being ever to have lived.

[00:14:18] He's going to be the mother generation of human beings. And because of his salience in the history of psychiatry, I think apocalypse delusions are well known.

[00:14:29] SEAN KING O'GRADY: We'll return in later episodes to further explore the grandiose flavor of apocalyptic thinking, and specifically the language of the tech community as they so frequently pronounce the world will be imminently saved or doomed.

[00:14:42] But before we move on, I have one more question for Joel. And it's because I want to be able to better understand my own fears about AI. Could this simply be a reflection of my own thinking? Is it healthy to be thinking about apocalypse and why do we think [00:15:00] about apocalypse?

[00:15:01] DR JOEL GOLD: You know, I think about anxiety as an internal state as opposed to fear.

[00:15:07] If a tiger is chasing me, I am afraid and I should be afraid and run away. Anxiety often feels, you know, the sort of unclear, why am I feeling this way? It's awful and I don't know why. But now we've moved from a place of anxiety to fear. It is appropriate to be afraid of these technologies. Someone comes to me and says, I'm not sleeping because I'm afraid.

[00:15:39] What do I do with that? Do I prescribe them something to help them sleep and say, you know, good luck with that? For me, as a psychotherapist, it's still most useful to focus on the real fear, again, founded in [00:16:00] reality, and see if there is some resonance with something more of an anxious place within them that might be specific to their history.

[00:16:12] their unconscious because I can do something with that. I can't, you know, do anything with someone fearing the end of humanity because of, uh, of AI or, or other technologies.

[00:16:25] SEAN KING O'GRADY: To paraphrase Joseph Heller's Catch-22, just because you're paranoid doesn't mean AI won't destroy the world. There's plenty of fear out there.

[00:16:33] Are these delusions? Are they based on reality or based on marketing hype? Are these warnings to be taken seriously? How do we think about any of this in a way that's useful? That's what I'm trying to figure out. This season is called AI and the Apocalypse. Let's make sure we're all on the same page about what those terms mean.

[00:16:53] AI is used really broadly right now. It is used to describe chatbots. It is the technology that's going to [00:17:00] replace your job. It's the software generating the graphics that create a clip you'll see on TikTok of the Incredible Hulk kissing Deadpool with the caption, "Hollywood is so cooked." It's in every commercial.

[00:17:11] It powers your gambling apps, your TV, maybe even your car. It's quite literally everywhere. But what does it technically mean? Here's how some of our experts described it last season. So I asked Nick Haber, a professor and researcher from Stanford, the epicenter of AI at academia, to, as simply as possible, just explain to us what AI actually is.

[00:17:37] NICK HABER: Oh, it's a toughie. Um, in, in some way, uh, it, it's a grand project to attempt to replicate aspects of human cognition or inspired by human cognition, make useful things that have some interesting form of cognition.

[00:17:56] SEAN KING O'GRADY: And now let's hear from Nate Sheridan, an ethicist and philosopher [00:18:00] at the University of Hong Kong and a research affiliate at the Center for AI Safety.

[00:18:04] I think you sit at a really interesting intersection of the sort of ideas and technology, so you might be the perfect person to explain, like, what is a large language model?

[00:18:14] NATE SHARADIN: Yeah. So I mean, the large language models that people are interacting with on the internet when they go to things like ChatGPT, you could think of them as, like, very carefully scaffolded LLMs.

[00:18:27] So large language models compress, like, very large data sets of text into, uh, something called weights, model weights. Uh, and then when you do inference over these weights, it become-- it's very good at predicting the next token. Uh, this is why some people have called it, like, fancy auto-complete. Um, so, uh, it's very good at, um, knowing how to, like, continue on, um- a particular, like, selection of [00:19:00] text.

[00:19:00] So chatbots like ChatGPT are just... think of them as LLMs that have been very carefully designed to be good at conversation.

[00:19:09] SEAN KING O'GRADY: What does AI mean to the people that are actually building AI? What does it mean to the people selling it? It's hard to pin down exactly. Here's tech writer Owen Higgins on how the term is used by AI advocates.

[00:19:23] What does the term AI mean if we're to take its biggest boosters and investors at face value?

[00:19:30] EOIN HIGGINS: I think that they would say that AI is a world-changing, earth-shattering technology that has the potential and likely the possibility to reshape the entire economy, if not the entire world. It's something that is going to both take your job and make your job easier.

[00:19:53] It is so powerful it could destroy the world, and it also needs to be invested in so that [00:20:00] it can be deployed for gains in productivity, fantastical gains in robotics and software, uh, robotics, whether hardware or software. But a lot of those ways are not going to really be seen in the way that the boosters talk about them, right?

[00:20:16] So boosters often talk about AI in a way that takes the public's ignorance about what AI really is and just kind of says, "This is a thing that can think and can do things, and so it's going to talk to you. It has a personality. It has its own agency." I, I think that it's a little more complicated than that.

[00:20:37] It doesn't have a personality. It doesn't have goals. It doesn't have desires.

[00:20:41] ALEX HANNA: Everything has been looped into this term AI. It's kind of why I put AI in quotes. Everything has been colonized by the term now.

[00:20:48] SEAN KING O'GRADY: This is Alex Hanna, sociologist, podcaster, and co-author, along with Emily Bender, of the essential book, The AI Con, further discussing how AI is used by the AI community itself in the [00:21:00] grand existential claims that justify its incredibly high valuations.

[00:21:05] ALEX HANNA: So in our book, The AI Con, we discuss two camps. One are AI boosters and one are AI doomers. And they like to pose themselves as diametrically opposed, and that they are effectively saying different things. So AI boosters say that, "Well, we need to build AI, and it's going to lead to an era of incredibly cheap goods, so much productivity that the working class is effectively going to not have anything to do.

[00:21:40] We're actually gonna have to institute universal basic income." AI doomers, on the other hand, say, "We're gonna make this thing, and if we don't teach it correctly, then it's going to kill us all." And this is exemplified pretty well by a few different authors. Eliezer Yudkowsky is kind of one of the main figures [00:22:00] here.

[00:22:00] He wrote a book literally called If You Build It, It'll Kill Us, or It'll Destroy Us. I actually f- don't know the exact title, but it's effectively this doomer, this doomsday thing, and he's this kind of patron saint of, of doomerism.

[00:22:14] SEAN KING O'GRADY: Alex is referencing If Anyone Builds It, Everyone Dies by Eliezer Yudkowsky and Nate Soares, which is a fascinating and controversial book Nate and Eliezer are associated with the Machine Intelligence Research Institute

[00:22:26] ALEX HANNA: These are effectively two sides of the same coin.

[00:22:30] So AI boosters say AI is a thing. It's a coherent thing. It is important to build. It is necessary to human progress, and it's inevitable, and, uh, it's gonna lead us to salvation. Whereas AI doomers say AI is a thing. It is, um, necessary to build. It is, uh, the next step in human progress. It's inevitable, and it's going to kill us all.

[00:22:57] MALO BOURGON: So my name's Malo Borgon. [00:23:00] I'm the CEO of the Machine Intelligence Research Institute. So we've been around for twenty-six years now. The way that some people, uh, summarize us is that we're the OG doomers. And the reason I kinda don't like that term, um, is because, you know, I think some of the people who, you know, are described as doomers, certainly some of the, the ones that are seen as maybe kinda like the most hardcore doomers, are also some of the folks who are earliest to thinking about, you know, w- whether we could build very powerful AI systems and what that would mean for the world, and potentially being, you know, e- very excited for the upside if we could get it right.

[00:23:32] SEAN KING O'GRADY: What is AI? What is artificial intelligence?

[00:23:35] MALO BOURGON: I like to, like, kind of, like, go back to when the field was founded, and people can read about it, but basically, the goal was there was a bunch of scientists who were kinda like, "These humans, we're doing this thinky thing. We're able to kind of, like, learn things in one domain and apply them to another domain and create plans and develop technologies and solve problems.

[00:23:53] Wouldn't it be cool if we could make computers do that? That would be super helpful. We could have them do a bunch of work and [00:24:00] labor, and maybe we could make them really smart, you know, maybe smarter than us, and then we could use them to, to help us do all the things." Like, most of the positive changes in human welfare throughout our history have been the product of us applying our intelligence to, to solving problems and developing new technologies.

[00:24:14] What if we could have AI systems do that? And so when I'm talking about AI I basically just mean in general AI systems that are capable of kind of generally doing the thinky thing that we're doing. The term AI kind of drifted from that meaning. They thought that they could maybe make some real progress on that in like a summer.

[00:24:30] Uh, turned out the problem was real hard and, you know, seventy years later, the field hadn't made that much progress on that kind of original goal. They had developed a bunch of systems that were good at specific things. You know, we could make AI systems that played chess better than a human grandmaster or were, you know, very good at image classification or other very narrow things.

[00:24:49] So AI kind of came to mean automating various types of intellectual or physical labor with systems, but they were very narrow. They were like very good at particular things. [00:25:00] And so the field kind of developed a new term to talk about the original thing, and they called that AGI, so artificial general intelligence, where the general is trying to point at this like general ability.

[00:25:09] In the words of, of Dario, the, the CEO of Anthropic, he wrote an essay, you know, talking about where he thinks this is all headed. And one of the ways that he described it is if we actually succeeded at building, you know, AGI systems the way that he means it, that this would be something like having a country of geniuses in a data center.

[00:25:25] The thing that I kind of try to imagine is, you know, what if we had AI systems that in some sense had kind of the ability to do most of the cognitive tasks that we could do That would kind of be AGI. So that could be math, that could be programming, that could be thinking strategically, that could be being very persuasive.

[00:25:45] AI systems, you can think of them kind of as weird alien minds that in some sense can do all the things that we can do, at least right now, you know, 'cause robotics is a separate question. How do we interact with the world? That's a little further behind than the, the frontier AI systems, [00:26:00] um, these LLMs. Uh, but the goal is not chatbots.

[00:26:03] The goal is, you know, agents. Um, you know, you're an agent, I am an agent, you know, I have some goal or some task at work, and I can just go off into the world and come up with plans and do work to try and accomplish things in the world. Um, but that's the type of AI system that they're trying to build. It doesn't seem like there's any reason why something like human-level intelligence is some sort of peak that we should be able to scale well beyond that.

[00:26:28] So you go from having a country of geniuses in a data center to having a country of super intelligences at a data center, which you can just think of as, like, just b- much, much better at all the things.

[00:26:38] SEAN KING O'GRADY: Basically, AI is a blanket term that refers to an expanding suite of computer processes. But throughout this series, unless referring to something specific like a chatbot, when we use the term AI, we're talking about this broad, almost abstract concept.

[00:26:54] We're talking about the AI of science fiction, which is becoming science reality, something closer to [00:27:00] AGI, or artificial general intelligence, or ASI, artificial super intelligence. Yes, those are two distinct things. We are talking both about the technology that empowers unconsented deepfakes and the research tool that may enable the greatest scientific breakthroughs in the history of humanity.

[00:27:17] Uh, this is, to paraphrase legendary Simpsons writer John Swartzwelder, potentially the cause of or solution to all of life's problems. It's the big thing that will change all things. So that's AI. Very easy, very clear, right? Now, the second part of our series title, Apocalypse. This word commonly refers to the end of the world or some great cataclysm that befalls humanity.

[00:27:46] It's typically not the total end of things, though, hence the endless amount of post-apocalyptic movies and TV shows. But it's bad news, like really, really terrible news. So what does the term really mean? I spoke with Professor Judith Wolfe from the [00:28:00] School of Divinity at St. Andrew's on the language of apocalypse.

[00:28:04] JUDITH WOLFE: The Greek apocalypse, apokalyptein means to unveil, to uncover, describing either the spiritual world that we can't see or the future that we can't yet see. So an unveiling of things that are ordinarily hidden to humans.

[00:28:21] SEAN KING O'GRADY: So apocalypse is an unveiling. It's the revelation of hidden truths. As with the Cthulhu passage we began the episode with, there's this idea that not all that is concealed should be unveiled.

[00:28:32] Here's Timothy Morton, a professor of English at Rice University, and most recently the author of Hell, In Search of a Christian Ecology, digging further into the word apocalypse and the great spilling of secrets.

[00:28:46] TIMOTHY MORTON: When we talk about apocalypse, are we really talking about something like a spoiler? The apo part of the word apocalypse is privative, right?

[00:28:57] It means on or non [00:29:00] or sort of off, something like that, right? And the calypto, calyptoin is to hide, right? So it's like you're unhiding, disclosing might be another way of putting it, right? What's interesting about that is that, of course, there is a kind of initial hiddenness, right? There is something that requires disclosure.

[00:29:22] When people say apocalypse, they often mean, well, now the gloves are off and the play is over, and we're gonna see who's been naughty and who's been nice, right? And someone's got a list, and they're gonna be sorting the sheep from the goats, and we're gonna get to see the reality underneath. Yeah. But what if what's being disclosed is the impossibility of that, right?

[00:29:47] What, what if actually when you think deeply about apocalypse, what you run up against is the possibility that this-- what is being disclosed is that you can never rip [00:30:00] the appearance off of something to see the workings of the real thing underneath?

[00:30:06] SEAN KING O'GRADY: I like the way Tim puts it. The apocalypse is like a spoiler.

[00:30:10] We're going to find out the great truths, even if the truth is that there is no truth. And by the way, I promise all of this relates to Grok, Claude, and ChatGPT. Just stick with me for a bit. But for now, there's another term worth defining, which I never really had to say out loud, I think, ever in my life until doing this podcast.

[00:30:29] Eschatological. Last season, we had to grapple with anthropomorphizing. This season, we're grappling with eschatological and eventually phenomenological, but we'll get to that later. Can you define eschatology?

[00:30:44] JUDITH WOLFE: Eschatology, uh, comes from Greek word eschaton, the last things, and so it's the study of the last things, and the last things really include both collective and individual ends and endings.

[00:30:57] So eschatology traditionally comprises [00:31:00] both the study of the end of the world, how we might conceive the world coming to an end, which also includes questions about the direction of the world as a whole.

[00:31:08] SEAN KING O'GRADY: What made you personally want to spend your life researching the end of things?

[00:31:15] JUDITH WOLFE: What else is worth researching?

[00:31:18] Um, I think what's so fascinating about eschatology is that it points us precisely to the, to the limits of our knowledge because this is all about things which neither speculation nor experience can really give us a clear sense of. These are not things that we can wholly answer either from out of experience or from out of rational deduction.

[00:31:40] It becomes a question of what are the constitutive limits of human knowledge? And in the face of these sort of radical limits of human knowledge about things which are nevertheless hugely significant for humans, how do we live in that? Uh, how do we live in and out of that kind of- [00:32:00] Unknowing

[00:32:01] SEAN KING O'GRADY: I love this idea.

[00:32:03] What else is really worth researching other than the limits of our knowledge? I wanna return to Timothy Morton now, because often when we're talking about the end of the world, we're talking about the end of one kind of thinking or the primacy of a certain kind of knowledge. Sure, sometimes we mean that Bruce Willis failed to accomplish whatever tedious thing he was doing at the end of the movie Armageddon, and the world explodes.

[00:32:27] But more often than not, we're talking about systems of beliefs and different ways of living. This is a big idea we're going to explore in greater depth later in the series. But when we talk about the end of the world, exactly whose world is ending? I think when I first became aware of your work, it was my understanding of something that you had said in your book Hyperobjects, which is that the world has already ended.

[00:32:55] What do you mean by that?

[00:32:56] TIMOTHY MORTON: Let's just cut to the chase. Stuff that's been [00:33:00] happening to everybody who isn't a white guy for a long time is now happening to white guys like me. We get to have the fun, unquote, of all the stuff that we imposed on everybody else. That's what I mean by the end of the world.

[00:33:17] SEAN KING O'GRADY: Tim might not be everyone's cup of tea.

[00:33:19] Then again, neither is anyone else in this series. But I found the most interesting ideas almost always come from the people who make you the least comfortable. So back to Tim

[00:33:29] TIMOTHY MORTON: I'm gonna say it again. I feel like I get paid to say terrible, rude things. Only very few incredibly insulated, very rich, powerful people can slightly enjoy it, um, as long as they don't have a conscience.

[00:33:42] And that's what I mean by the end of the world. It's, it's the end of the idea of nature as a kind of stage set on which human being meanings can be played out to some kind of audience, possibly also of, of the same human beings, right? And, um, we like to talk [00:34:00] about, "Oh, it's gonna be the end of the world.

[00:34:02] It's gonna be the apocalypse." But the end of what, right? It's basically the end of a whole bunch of projects that, that never were that great in the first place. It's a little bit like the, the end of the world as a concept and, and the end of the idea that the, that the world is ending, well, it's, it's gonna be this incredibly long-term drag for everybody until such time as people decide to, to treat each other nicer, to use the technical term

[00:34:32] SEAN KING O'GRADY: I've spent some serious time thinking about this, that the end of the world is the end of the idea that the world is ending. It kinda twists your brains in knots for a minute, but the idea here is that the great unveiling is that there's nothing there. This is the black seas of infinity, just endless nothing.

[00:34:52] We'll return to the idea later that talking about the world ending is also exposing biases about what kind of [00:35:00] world is worth saving. But I do wanna clarify our own point of view a bit. In this series, we're specifically discussing the artificial intelligence and system of beliefs that originates in Silicon Valley.

[00:35:13] So when we discuss the end of the world, it's often going to be through the lens of what we understand having been raised in and steeped in Western culture, philosophy, and religion. This isn't to ignore or minimize the global influence and contributions to Silicon Valley contemporary American culture and America itself.

[00:35:30] It's to find an entry point into this Byzantine subject matter and to contend with epistemologies most regularly represented in our news, media, and popular culture. The differences between the guiding philosophies of Silicon Valley AI culture and Chinese AI culture, for instance, are worthy of their own podcast series, truly.

[00:35:50] So I hope someone makes that, but we will only touch on it briefly as a counterpoint of sorts in our series. Perspective matters a great deal in these [00:36:00] discussions because throughout history, plenty of people have thought they were living through the end times. In fact, if our research is remotely accurate, most people during most times have felt the end of the world was at least possible in their own lifetime, if not likely.

[00:36:15] And in an apocalypse as a human extinction kind of way- Everyone's always been wrong, as evidenced by the fact that we're all still here talking about this. In the next episode, we're going deep into the history of apocalyptic thinking, but there's one really important question I want to foreground now: Is this moment in time different?

[00:36:34] Everyone always thinks their special place in history is unique, but like right now, I don't know, maybe it is. In terms of the world ending, we're absolutely in unprecedented territory and have been for some time, and there's one big explosive reason why. The Trinity Test, the first real world test of the atomic bomb, which took place just over 80 [00:37:00] years ago.

[00:37:00] This is something we'll reference regularly throughout this series. Here's Professor Judith Wolfe again explaining how our ability to destroy humanity at the push of a button from that moment onward has affected the field of eschatology.

[00:37:14] JUDITH WOLFE: I think the last three-quarter century or so has been unprecedented in the history of human thought simply because until then, if we talked about the end of the world at all, we talked about it either in a very abstract scientific register where we speculated about heat death or about, you know, meteors and so forth, or if we spoke about it in a religious register, it was really the purview of religions to think about the end of the world.

[00:37:41] And so It was really all of a sudden that something like the end of the world came within the reach, uh, of human possibility. And so that raised a lot of questions among theologians, of course.

[00:37:54] SEAN KING O'GRADY: J. Robert Oppenheimer, scientific director of the Manhattan Project and overseer of the [00:38:00] Trinity test, famously quoted the Bhagavad Gita, a collection of Hindu scriptures and epic poems, as he watched the very first mushroom cloud rise into the clear desert sky on July 16th, 1945 at 5:29 AM Mountain wartime, stating, "I am become Death, destroyer of worlds."

[00:38:22] This is eerily similar man-as-God framing to the language we hear from AI CEOs today. Even the name Trinity invokes religion, specifically the Christian idea of the Father, the Son, and the Holy Spirit. And one quick aside, a more modern translation of the Bhagavad Gita passage is, "Look at me. I am time itself, and I must one day destroy your world as I have always done."

[00:38:51] I find this really interesting because this is a Westernization of an Eastern concept that changes something [00:39:00] cyclical to give it a clear end, and we actually see echoes of this in Western and Eastern views of AI. All right. Considering I brought this up twice now, maybe we will cover the differences between ideas, philosophies around AI between the East and the West in this show.

[00:39:17] But for now, back to Trinity. Much like stated concerns from today's business leaders, the Manhattan Project scientists weren't sure mankind would even survive the first test of this world-changing bomb. There was sincere concern that the bomb could ignite the nitrogen in the atmosphere and trigger a chain reaction that would propagate and consume the entire Earth.

[00:39:39] Fortunately, that hasn't happened. Not yet, anyway. Yeah, and by the way, there's a small movie Christopher Nolan made about Oppenheimer and the Manhattan Project called Oppenheimer if you're interested in learning more. I'd actually read the book it's based on, the brilliantly titled American Prometheus, maybe, like, 10 years ago, but while [00:40:00] watching Nolan's movie about 1945 in 2023, I just couldn't stop thinking about AI.

[00:40:06] It's wildly relevant, and even more so today in 2026 than it was just a few years ago, and I think that says something about the speed at which things are happening. One of the things we're looking at is the potential likelihood of, of an AI apocalypse, which could take any number of forms. So what has it been like for you as you've been in this field of study as a new potential apocalypse agent has come into being?

[00:40:33] Has this been a, a, another radical shift, or is it sort of an evolution of the previous one?

[00:40:38] JUDITH WOLFE: I mean, to some extent, this is a new moment in history. Clearly, we have never created something that has put us in the position both of creator in some way and the potential inferior of, uh, of something that we've created.

[00:40:53] But on the other hand, uh, I think this is a version of something that has happened over and over [00:41:00] again in Western history. I think studying Western history, one of the things that we see as a pattern emerging again and again is that one of the basic human needs being a sense of orientation in the world, a sense that we understand the world around us, such that we can act in it, such that we can orient ourselves in it.

[00:41:18] And so whenever in history there have been times when multiple forms of crisis have all come together, political crisis, intellectual crisis, crisis, in other words, of what we understand as knowledge, economic crisis, natural crisis, when things have seemed so bad that they've threatened our very understanding of the world, those were always times when eschatological or apocalyptic thought has surged in the population.

[00:41:46] Partly because things have seemed so bad that they can only betoken one thing, which is the end of the world. Um, and partly because apocalyptic thought, eschatological thought can, and again, I think that this is erroneous, but it's [00:42:00] clearly how it's often functioned, can give people a sense that they have this almost transcendent, supernatural key to the interpretation of what's going on, and therefore have a sense or have the ability to make sense of things where sense is really unavailable to them, where things just don't make sense anymore, where there's no firm ground under their feet.

[00:42:22] And then the assignation of different roles, you know, set roles, the Antichrist, the Savior, et cetera, can give people a sense that they can orient themselves in a world that's otherwise falling apart.

[00:42:33] SEAN KING O'GRADY: In very broad terms, it's essential to think of apocalyptic discourse being pre- and post-Trinity.

[00:42:40] There's a time when the end of the world was the domain of gods or forces beyond our control, and there's a time when the end of the world can be engineered, and unfortunately quite easily by humans. Confused, angry, jealous, selfish, joyous, brilliant, and beautiful humans I asked Ed Simon, a [00:43:00] professor at Carnegie Mellon University and expert on all things apocalypse, about the historical importance of Trinity.

[00:43:06] ED SIMON: The Trinity test, the, uh, Manhattan Project itself, in terms of the cultural history of apocalypticism was a massive sea change. And you do see it in evangelical Christianity that kind of embraced understandably the possibility of nuclear war as a means of the end of the world being facilitated, right?

[00:43:25] Uh, but I think in secular thinking as well it made a huge difference because what the possibility of nuclear war finally manifested was the actual chance of humanity being able to enact a type of apocalypse on the Earth in a way that they could not before, right? Like people could talk about the end of the world up until 1945, but we couldn't do it ourselves.

[00:43:49] Whereas I think once you have, I mean, I don't know, there's like 3,000 nuclear warheads spread across seven, eight, nine different countries that have nuclear capability. Maybe it's higher than that even.

[00:43:59] SEAN KING O'GRADY: Oh [00:44:00] It is. 3,900 warheads are currently actively deployed, around 2,100 of which are on high alert, ready to launch on short notice.

[00:44:09] This means push of button, boom, 2,100 world-ending bombs. But there are believed to be over 12,000 total nuclear warheads in the world, and this is actually down from a total of 70,000 during the Cold War. This is the last encouraging news you'll receive over the course of our eight episodes. I apologize in advance for any sleeplessness, rumination, or unwanted weight loss or gain you experience as a result of listening to this series.

[00:44:35] ED SIMON: I mean, in 20 minutes, the whole world could be, like, reduced to radioactive ash. That is a difference, uh, from what the world was like in 1944 versus what it is like every year after that, and I think that can't help but have a tremendous cultural change. Uh, I mean, we have all... Unless you are, um, you know, well into your 90s, you cannot remember a world where that was not a possibility, [00:45:00] and I think that that hangs over everyone's head if they're being honest about it and they're being aware of it.

[00:45:06] And sometimes you have it, um, kind of, um, shunted into traditional religious rhetoric and language, which is kinda ready-made for talking about those things. But certainly you've had any number of secular explorations of what, like, a nuclear apocalypse itself would mean. But I think it's foundationally the most important event in recent human history because of how it changed what it means to be a human in such a fundamental way.

[00:45:31] SEAN KING O'GRADY: You wrote about two cultural responses to nuclear war. The c- there was a Christian fundamentalism and an Aquarian counterculturalism. Mm. Can you tell me a bit about that?

[00:45:43] ED SIMON: I ask what are the different ways in which people could react to this reality that we now live in a world where human beings are capable of destroying the world in a matter of minutes, right?

[00:45:54] Where we've kind of, like, immanentized the apocalypse in some ways. And so there's a, a Christian [00:46:00] fundamentalist reaction to that, which I think is predictable and makes sense. But then I also make the argument that the kind of, like, New Age Aquarian 1960s ethos was also a reaction to that. Now, you're talking about people who were children when, uh, Hiroshima happened, and now they're adults, and I think that there is a kind of way in which that was a manifest reaction to The horror of, uh, what nuclear war portended and that you had a new type of spirituality or religious exploration that reacted to that in a way that was different to, but kind of equivalent to, the Christian fundamentalist reaction towards it.

[00:46:40] But I can't imagine, like, the flower power generation existing in a world without nuclear weapons in some ways, right? Uh, and I don't think it's often spoken of necessarily in that kind of way, but there were, like, clear millennial and apocalyptic overtones to how a, a lot of people at that time talked about a kind of unveiling.

[00:46:58] I mean, tune in, turn in, [00:47:00] drop out from Timothy Leary is an unveiling, right, in a lot of ways, uh, as an ethos. So there's a kind of a apocalyptic changed consciousness idea that is, uh, im- implicit in that.

[00:47:11] SEAN KING O'GRADY: Do you think we'll see a similar philosophical or, or religious schism or reaction to the current moment with AI and all the other compounding, uh, potential apocalypse agents?

[00:47:22] I hope so.

[00:47:23] ED SIMON: I feel like we've got a kind of a crisis of meaning in some ways. There's a lot of embrace of irrational, kind of dangerous beliefs that people adhere to because of that sort of crisis of meaning. So it would be, I think, fascinating to see a, a kind of a, a renewal that asks some fundamental kinds of questions and maybe supplies a type of meaning for people or allows them to explore what it means to be a human being in a way that isn't just tied into kind of transnational capitalism and the reactionary politics that supports that.

[00:47:56] I don't know what that would look like necessarily.

[00:47:59] SEAN KING O'GRADY: I don't know [00:48:00] what that would look like either. Dorian Lynskey literally wrote the book on stories we tell about the end of the world, and what's so interesting to me is how he distinguishes the way we talk about death and destruction with the way we talk about paradise or utopia or the time when we will all start being nice to each other, as Tim Morton said, or however you personally wanna think about it.

[00:48:20] DORIAN LYNSKEY: As I was writing the book over and over again, you would see versions of either Revelation or the flood. You would see it in apocalyptic fiction and post-apocalyptic fiction and catastrophe fiction and on and on and on. And then the way people talked about fear of civilizational collapse, the way survivalists talked, the way people talked about overpopulation, the way people talked about AI.

[00:48:43] It's just, it, it never stopped, and I was, "Oh God, this is so deep in our brains," and it doesn't matter if you have never had any religious faith. These narratives are there. You know, they manifest themselves in politics and in science and in [00:49:00] art. We cannot get away with that idea of the great everything is degenerate and failing.

[00:49:07] And then there will be some cleansing violence and we will be reborn in this life or the next. And we keep coming back to that. And it keeps turning out that the destruction is an awful lot easier than the rebirth. That's the problem. It's really, really easy to knock things down and it's very, very hard to rebuild them.

[00:49:33] And that's the problem with apocalyptic thinking is that even in the book of Revelation, the violence and the war and the fire and the demons take up an awful lot of that book. And then literally in the final pages, it's like, oh, and then it's all fine. Then there's paradise. And you're kind of like, is paradise on earth or in heaven?

[00:49:55] And it doesn't really say. What does it look like? What do people do there? Don't [00:50:00] really know. Like the writer of Revelation does not care.

[00:50:05] SEAN KING O'GRADY: Let me interrupt Dorian for a second to confess, Father forgive me. Despite eight years of Catholic school, I just read Revelation in its entirety for the first time while doing research for this project.

[00:50:17] And man, it is a wild read. Deeply psychedelic. Gives the Tibetan Book of the Dead a run for its money. 10-10. Would recommend.

[00:50:26] DORIAN LYNSKEY: He will go into enormous detail about the torture and Satan and the description of the beasts with like lions heads and serpents tails and all of that stuff he loves. And when it comes to describing utopia and peace and happiness, he just kind of like tunes out.

[00:50:45] It's very lovely lines, big beautiful lines at the end of that book. But he doesn't want to describe that. He doesn't know what that's going to look like. And that's the problem. We never really do. What does the AI utopia look like? People aren't really, we don't really [00:51:00] know. We're not very good at imagining the perfect world, but we're very good at imagining destroying the world.

[00:51:08] SEAN KING O'GRADY: We're not very good at imagining the perfect world, but we're very good at imagining destroying the world. Elon Musk recently tweeted that the future of AI abundance is such that every single person will be able to live in their very own penthouse apartment. Think about it for a minute. This is notable for a couple things.

[00:51:33] One, the hilarious logistics of every single person on the top floor of their very own building. And two, it's kind of lame. Is this really what people want if they have Elon's proposed universal high income? Personally, I'd rather be at that much memed cave rave from the second Matrix movie with like 10,000 other people who never have to go to work again.

[00:51:56] Seriously, if the best case scenario [00:52:00] of AI abundance is that everyone lives in isolation at the top floor of different buildings, and the worst case is the end of humanity, and we're just blazing forward, damn the consequences, I'm really happy that it's so easy to make fun of, but it's actually scary, too.

[00:52:17] But I truly, I find this stuff genuinely frightening, and a lot of people do. That same passage about Cthulhu that we began with is also cited in the book Bunker by Bradley Garrett, who we'll hear from directly later in this series. The focus of his book is doomsday prepping, and of this passage- Garrett writes, "Many of the preppers exhibit a clear sense of dread with respect to our collective trajectory as a species.

[00:52:45] Dread differs from fear, both because it is about the future rather than the present, and also because it stems from a danger not immediately present or even discernible. In other words, fear has an [00:53:00] object. Dread does not. And it is, I believe, the dominant effect of our era." Wow, that's heavy. Dread might also be a way to categorize our relationship to AI.

[00:53:14] It is promised to be something epic-defining, but we're not quite sure what. It could elevate our species, but we don't know yet exactly how, or it could mean the end of all civilization and life on Earth through means which we can't really articulate, but sound really bad. I just need to throw this out there too before anyone accuses me or this show of being zealously anti-AI.

[00:53:39] I'm not. I'm excited about the potential of AI. Seriously. I wanna live in a Jetsons future as much as the biggest AI booster. I would love for us all to have universal high income that would allow for my kids to spend their entire future on a French Riviera holiday if that's what they want. But I'm also truly terrified their [00:54:00] future could end up much darker as a result of us incorrectly implementing AI into our world.

[00:54:05] And, and it's really the uncertainty, uh, that we know something's going to change, but we don't know how, we don't know when, and that's what causes this dread. This is what we're digging into over the next episodes. What do we do with our dread?

[00:54:21] Ignore it because AI might end up being false hype or a panacea for the world's problems?

[00:54:26] Take it seriously because the means of mass destruction are so readily available, and quite frankly, we're really lucky that in seventy-five years of having access to all these nuclear weapons, we haven't blown each other up. Do we find solace in histories of the apocalypse or the millennia of art we've created to reckon with these thoughts?

[00:54:44] And in case this all sounds too abstract or dark to listen to while working out or doing your dishes, we're also talking about how demons are literally real Here's Timothy Morton again on artificial intelligence and actual demons

[00:54:59] TIMOTHY MORTON: Sort of [00:55:00] intuitive way that people have started to talk about it as a sort of scary kind of demonic, right?

[00:55:06] Demonic literally is ano- it's another Greek word, right? It, it has to do with influences that we cannot perceive, right? Like you're being affected by something that you can't point to, which is why in, you know, the old-fashioned days of Stripe, um, I'm old enough to remember the early email software. There was the mailer demon was the thing that you'd see very, very often.

[00:55:26] When you get the email, it would say, "Mailer demon this and that," and blah, blah, blah. The demon being a process that's operating in the background to send you the message, right? Above and beyond angels versus demons, it's that what is demonic is influences that cannot be perceived, that are nevertheless influencing one, right?

[00:55:45] And so to that extent, sort of thinking of AI as a kind of demonic force is absolutely accurate without any moral necessarily or aesthetic kind of judgment about that. But just [00:56:00] building this satanic version, demonic version of what is already a problem. You know, there are so many stories about the devil.

[00:56:08] You know, the devil comes along and goes, "You, you can... If you do s- sign your soul away on this piece of paper here, then I'll give you seven wishes." They usually don't say that part out loud very loudly, right? "I'll give you seven wishes." And you wish for the thing, and you get exactly what you want, right?

[00:56:24] The devil or the magic fish that popped out of the ocean, right? The magic fish pops out and goes, "You can have three wishes." And the guy goes, "Oh, I want to see my son back from the war." And the fish goes, "Okay." Boom, along comes a coffin with the son in it, dead. Yeah. "And I want 100 pieces of gold." "Oh yeah, sure.

[00:56:42] That's compensation 'cause your son got killed." And the only sensible thing to do then is for the guy to wish that he never had the wishes in the first place, right? And this is the situation we're in. Let's build the perfect servant. What could go wrong? You know, the... Every story about the devil in the world is the story of the [00:57:00] perfect servant who then becomes the ultimate master by doing exactly what you tell them to do.

[00:57:06] And this is what software does, right? It does exactly what you tell it to do, no more, no less. And there's the problem.

[00:57:13] SEAN KING O'GRADY: On the next episode of Suspicious Minds: AI and the Apocalypse, where does our doomsday obsession come from? Why are we psychologically predisposed to fantasizing about our own demise?

[00:57:24] It's the origin story of the end of the world.

[00:57:27] KELLY BULKELEY, PHD: There is a darker temptation of apocalyptic thinking that can arise from frustration and powerlessness, a sense that this world is so corrupt, it's just irredeemable, and the only way it can be ever changed or purified is by just, you know, burning the whole thing down.

[00:57:45] SEAN KING O'GRADY: If you're enjoying Suspicious Minds, be sure to follow or subscribe. And if you really like the show, make sure to rate us on Apple Podcasts and like on other platforms. If you want to learn more, you can go to the show notes where you can find links to our socials along with links to [00:58:00] works from our contributors.

[00:58:01] They appreciate your support and so do we.