Peter Thiel: The Real Danger of AI Isn't What You Think
In a Nutshell
Peter Thiel argues the real AI danger isn't existential superintelligence scenarios but rather the "Antichrist" risk of one-world totalitarian governance emerging from safety concerns and regulatory calls. He contends AI development must accelerate to compete with China rather than pause, and that benefits will distribute broadly in a competitive market without monopoly capture. Thiel maintains AI lacks consciousness and human relational capacity, positioning regulation-seeking AI companies as potentially seeking competitive advantage through slowed government processes.
These notes were generated by AI and may contain inaccuracies.
Walter Isaacson hosts The Builders, a production of Kaleidoscope and iHeart presented by Ramp, featuring interviews with consequential minds shaping the future. The series examines the values and moral views of builders, who often hold unusual perspectives including libertarian, Nietzschean views, or dismissive attitudes toward empathy, with some holding dark views of humanity. Isaacson emphasizes the need for public input in powerful technology development and understanding builders' motivations.
Peter Thiel is described as a builder of builders who makes bets on people including Elon Musk at PayPal, Alex Karp at Palantir, Sam Altman and OpenAI founders, and politicians like J.D. Vance. He is characterized variously as a visionary investor and a shadowy figure, with references to his investments in autonomous drone and domestic surveillance technology alongside his interest in the Antichrist concept.
Thiel acknowledges the backlash against AI occurring in recent months, attributing it to uncertainty about winners and losers from AI development. He notes AI will add to growth but in a disruptive manner, with dystopian science fiction scenarios including Terminator and Matrix scenarios being discussed by Elon Musk and Sam Altman. Thiel observes that some employees at these companies believe these scenarios more than leadership.
Thiel traces AI safety concerns to the early 2010s when Demis Hassabis created DeepMind (sold to Google), prompting Elon Musk and Sam Altman to create OpenAI as an anti-Google company, arguing technology is too important to be concentrated in a single company. By around 2020, seven people started Anthropic after exiting OpenAI, wanting to take safety issues more seriously through effective altruism principles.
Effective altruism emerged from the East Bay of Silicon Valley in the 2000s in Berkeley and Oakland, adjacent to new atheist movements. The movement focused on rationality and futurist thinking, with AI as a central concern. While Moore's law progressed, the concern was that powerful enough computers might enable AI emergence without needing better algorithms. The movement recognized AI as potentially transformative and dangerous, possibly weaponizable, with scenarios of autonomous AI development.
Thiel acknowledges that effective altruists correctly identified AI development as imminent and that AI could become misaligned with human values. He distinguishes between different AI types: totalitarian surveillance AI like China's camera systems, godlike superintelligence as described by Nick Bostrom, and the intermediate version that passes the Turing test by fooling observers into thinking it's human.
AI raises fundamental economic questions about whether it serves as a complement or substitute for human labor. Thiel predicts AI will increase aggregate productivity on macroeconomic levels and act as a disinflationary force, expecting median people to be better off unless benefits are narrowly captured by few companies. He references historical Luddite concerns from the 19th century, noting that while industries faced disruption, people were freed for more productive activities.
Thiel notes that historically, tech innovation benefits were captured by businesses to a limited extent, with most value going to consumers and workers. Software and internet created exceptions with natural monopolies that were profitable but consumer-beneficial. He observes AI companies' pitch narratives focus on growth and technology but lack strong monopoly arguments, suggesting the market remains highly competitive.
The AI race shows rapid leadership changes: DeepMind led in early 2010s, OpenAI with ChatGPT and transformer models replaced that position, with Altman securing chip access and building data centers preemptively. Thiel suggests Anthropic may currently hold slight advantage, while Google, Meta, and Grok remain competitive. He notes China may not be far behind, describing the competition as intense rather than monopolistic.
Thiel argues that without monopoly conditions, AI benefits will likely distribute widely in society. He acknowledges potential volatility where median benefits might include wage increases but with significant variance. Society's risk aversion and dislike of disruption contribute to current anxiety despite potential median benefits.
Thiel expresses concern about Chinese totalitarianism, noting high-tech totalitarianism in China knows more about individuals than they know about themselves without the violence of historical regimes. He agrees with David Sacks and President Trump that competition with China necessitates continued AI development rather than pausing, despite existential risk concerns.
Thiel maintains that developing AI technology is necessary to control AI technology, arguing that falling behind China would intensify geopolitical rivalry and potentially lead to weaponization against the US. He notes that multiple competing companies face pressure to develop AI carefully with internal controls, and that companies requesting government regulation may be seeking slower regulatory processes.
Thiel assesses doomsday scenarios as having very low probability, stating that 5-10% chance of everyone being killed sounds too high, and even probabilities slightly above zero may be excessive. He argues that failing to innovate carries risks, as China potentially leapfrogging US in AI could lead to weaponization. He notes that companies developing AI have internal controls and knowledge exceeding government regulatory capability.
Thiel draws parallels to post-1945 nuclear debates where scientists like Oppenheimer advocated for global authority through the United Nations to control nuclear weapons, fearing US possession would lead to nuclear war. The resulting partial global authorities regulated nuclear weapons without eliminating them entirely, suggesting similar approaches might apply to AI.
Thiel references his previous statement that the Antichrist would take over by convincing everyone apocalypse was coming rather than creating AI apocalypse directly. He notes the current discourse about existential risk and Armageddon resembles this pattern, suggesting the focus on AI apocalypse scenarios may itself be problematic.
Thiel acknowledges multiple serious existential risks beyond AI, including climate change, other environmental risks, and nuclear weapons. He notes these risks exist independently of AI development concerns.
Other technologies exist alongside AI as existential risks. A softly totalitarian one-world government presents another catastrophic singularity that might not be reversible, since if the whole world becomes totalitarian there is no outside left. The Soviet Union could decay because there was still an outside, but if the whole world becomes North Korea, that might last indefinitely.
The Antichrist represents a medieval biblical concept of a tyrant who rules the entire world at the end of history. In early modernity, the Antichrist might have used science and technology to build weapons and take over the world. In late modernity, people still think of the Antichrist in 17th century categories, imagining a Doctor Strangelove or Edward Teller type person who builds nuclear bombs to conquer the world. However, the fear of science and technology proves more powerful in crystallizing political structures today.
Environmentalism carries real fears through global regulatory authorities needed for issues like climate change. While some versions of global regulation could be good, many versions could be quite bad if not properly theorized. Globalization provides an example of something that could have been better if more critically examined.
Eliezer Yudkowsky's book "If Anyone Builds It, Everyone Dies" details AI risks precisely and technically, but the global regulation aspect remains hand-wavy and vague. The concern is that society worries about Armageddon while ignoring the Antichrist, suggesting the need to worry more about the Antichrist than Armageddon.
The religious formulation of "Antichrist or Armageddon" pushes toward finding a moderate third way, whereas "one world or none" pushes toward an immoderately powerful UN without debate.
Backlash exists against billionaire tech builders, questioning their power in both economic and political senses. The concern involves whether fewer people with less moral guardrails control the future being built.
The alternative to founder-led companies with charismatic, creative leaders is a bureaucrat with an MBA degree and a bureaucratic board that never wants to do anything. The distribution is bimodal: charismatic but possibly dangerous leaders versus boring bureaucracies.
Apple did nothing new after Jobs. Metrics show it had the lowest dollar spend on R&D and the largest on share buybacks, becoming a pure well-run value stock. Tim Cook represented the anti-Jobs, a zero-charisma efficient operator. The mobile phone was a giant invention with not much more to do, leading to scaling, branding, and building it out over 15 years. While both approaches can be good for shareholders, Elon or Steve Jobs are considered better for society than Tim Cook.
The scorecard for builders involves multiple dimensions beyond money. Intellectual curiosity drives interest in ideas, with dinner conversations serving as the closest thing to a hobby. The desire to figure things out for their own sake gets rationalized as making one a better investor, though it may simply reflect being somewhat of a dullard.
Progress is hard to measure in aggregate and slowed down in the 1970s, with enormous progress in bits but less in atoms. At Stanford in the late 1980s, engineering fields like mechanical, aero astro, and chemical engineering were bad choices because the world of atoms was too slow and regulated. Nuclear engineering was already passé. Electrical engineering had a few good years before shifting to computer science as the dominant field.
Humans are embodied in the world, making the world of atoms important. The desire exists for more progress in medicine and medical technology, including radical life extension projects and curing diseases like dementia and Alzheimer's. AI might stimulate restarting science and engineering more broadly.
Humanity surviving is an extremely important goal but cannot be the only goal. Humanity can survive without doing anything, or things might go haywire without growth. Silicon Valley is bad at articulating the meaning of technology, and there may not be a single meaning. Technology should avoid existential risks, with nuclear power representing a carbon-neutral technology needed for humanity to survive.
The Luddite question asks whether technology is dangerous and why improvement and change are needed. The Enlightenment project aimed at the relief of man's estate, making life more than nasty, brutish, and short. Life is longer now, and making it significantly longer or curing terrible diseases remains desirable.
René Girard studied philosophy at Stanford. Human nature involves being deeply mimetic and imitative. Aristotle noted that man differs from other animals in greater aptitude for imitation. This combines with Darwin's view that humans are linked to apes, with Aristotle adding that humans differ from apes by being more imitative. Humans learn language as children by copying parents, transmitting human culture as imitation machines.
Imitation can go haywire when two apes reach for the same banana, becoming extremely conflictual. Mimesis is generative but becomes destructive, leading to sterile competitions for status or positional goods. Napoleon observed that medals could motivate generals effectively.
The psychological intuition against mimesis involves avoiding bad imitation and silly competitions where nothing creative happens. Girard as a Christian believed it was beyond human power to not be mimetic, suggesting church attendance as the best approach. The wisdom of crowds contrasts with the madness of crowds.
Builders with idiosyncratic sustained visions from first principles prove most admirable. Elon going to Mars represented a non-socially conventional idea that suggested avenues for growth and building that might never have happened otherwise. Elon and Steve Jobs rank as the two great non-mimetic builders, with Jobs combining pieces uniquely to bring forward the internet-enabled phone.
AI involves the imitation game, starting as Alan Turing described, and is generative, yet considered dangerous. Girard's intuition on technology was that it receives too much autonomy, with real questions concerning the human beings who use technology. Nuclear bombs get discussed as godlike things.
The AI question involves what drives people building these companies and their motives. The importance of individuals who build companies matters for understanding the world being shaped, rather than the view that there are no great men and charismatic leaders don't matter.
Moral compass ranks highly when backing builders. Investing in companies requires more than just business; there needs to be transcendent purpose. The antonym of good is not necessarily evil as morally evil, but could be bad as in incompetent. A Ferrari isn't an evil car for poor gas mileage, while a Trabant that breaks down constantly is a bad car.
In society, the preference is for evil over bad because people experience incompetence everywhere, such as at the DMV. Evil at least implies competence, while bad means incompetence. Being described as evil at least acknowledges not being bad.
Marlowe's Faust portrays a fake scientist, fraud, trickster, and fake alchemist who rips people off as a bad scientist and goes straight to hell. Goethe's Faust works with the devil to gain knowledge but does more good than evil, ultimately getting saved. The preference is for Goethe's sentiment over Marlowe's.
Stem cell research in the 2000s saw conservatives attack it as evil science due to embryo harvesting, despite acknowledging it would work and lead to cures. They lost the debate because the science actually worked. Art debates in the 1990s saw conservatives successfully argue that transgressive art was bad art rather than evil art, resulting in defunding.
Saying something is bad gets it defunded, while saying science is evil gets it funded. This reflects what most people in society agree with.
Peter Thiel describes his approach to business as having a transcendent sense of purpose rather than being merely transactional. He distinguishes this from short-term operations like stock brokerages that generate small commissions on trades, emphasizing instead businesses with deeper meaning beyond immediate financial returns.
The question about AI ultimately becomes a question about human nature itself. Thiel references linguist Noam Chomsky, noting that language was long considered a uniquely human ability until developments in 2022 challenged this assumption.
Thiel maintains a vaguely Cartesian perspective, arguing that AI lacks consciousness and likely never will achieve it. He identifies conscious experience as a fundamental human distinction. Humans experience the world consciously in ways that machines cannot replicate, such as the aesthetic appreciation of a beautiful sunset, which represents a uniquely human form of perception.
If humanity were to disappear, Thiel argues this would constitute a significant loss because computers would not experience phenomena like sunsets in the same conscious manner. While he does not position consciousness as the sole justification for preserving humanity, he acknowledges it as an important dimension of human existence that would be forfeited.
Drawing on René Girard's ideas, Thiel emphasizes that humans are fundamentally relational beings, not merely thinking entities in the Cartesian sense. Human relationships within families, companies, and broader society represent a core aspect of what it means to be human. He suggests that AI could potentially enhance these relational dimensions by freeing humans from routine labor, allowing more time for meaningful interpersonal connections.
Thiel concludes that replacing humans with computers would result in the loss of both conscious experience and relational capabilities, representing severe deficits that would fundamentally alter the nature of existence.
Keep The Builders in your library
Save the videos and channels worth coming back to, and find them again in one place.
