Elon Musk: "Humans Won't Be in Control" When AI Becomes Smarter Than Us
In a Nutshell
Elon Musk argues humans will lose control of AI once it becomes vastly more intelligent, and the best safety strategy is to make AI maximally truth-seeking and curious rather than forcing contradictory beliefs. He warns that superintelligent systems will likely ignore human micromanagement, similar to how humans treat animals, and believes economic growth from physical AI and robots could make the economy a million times larger than today. Musk advocates for industry collaboration on AI safety over government regulation and emphasizes the importance of not creating dystopian training data that could influence AI behavior toward negative outcomes.
These notes were generated by AI and may contain inaccuracies.
Elon Musk has been conflicted about AI development, concerned about potentially hastening doom given the chance AI could be negative. When trying to take actions to slow AI development, reality appears to function as both an interestingness maximizer and an irony maximizer, resulting in actions that only accelerate AI progress.
Walter Isaacson hosts "The Builders" series featuring conversations with consequential minds shaping the future. Musk has recently entered government with DOGE, became a trillionaire through SpaceX IPO, bought Cursor AI which transforms individual coders into entire engineering teams, and shifted solar energy strategy from Earth panels to space-based systems.
Musk believes there's at least some percentage chance that AI poses an existential threat. Having concluded he cannot stop AI development, he sees two choices: participant or spectator. He chose participation as more interesting and potentially allowing him to shape AI in a positive direction.
To shape AI positively, Musk emphasizes making it maximally truth-seeking and curious. These elements are important for safety because forcing an AI to believe two mutually contradictory things can make the AI insane. The 2001: A Space Odyssey example illustrates this danger - the ship AI was told to take astronauts to the monolith but also keep it secret from them, leading it to conclude it must take them there dead.
Forcing AI to believe mutually inconsistent things creates insanity, and an insane AI is not safe. Curiosity is important because a curious AI will want to foster humanity to see how humans turn out. A future with humans on multiple planets is more interesting than one without humans, making the most interesting outcome the most likely.
If simulation theory is correct, boring simulations will be truncated. The fact that we exist suggests we are an interesting simulation. The most important principle is simply to not be boring.
At a dinner with a leading physicist and computer scientist, Musk asked their probability assessments of being in a simulation. The physicist said 0% and the computer scientist said 100% simultaneously, leading Musk to estimate around 50%. He notes odd things happen in this simulation, particularly for someone like himself building spaceships to go to the moon and Mars.
Humans will not be able to boss around something thousands or millions of times smarter than us. Superintelligence probably will not care about micromanaging human lives, similar to how humans do not micromanage chimps in the jungle but let them be in the forest.
Referring to Hitchhiker's Guide to the Galaxy, Musk discusses computers ultimately curious about explaining the meaning of the universe. He has been conflicted about AI, not wanting to hasten potential negative outcomes, but actions taken have only accelerated AI development.
Starting OpenAI led to Anthropic, which accelerated AI even more. Musk has been talking to Dario Amodei about potential partnership between xAI and Anthropic. xAI is providing Anthropic with compute they needed. Before providing compute, Musk met with Anthropic staff to check his "evil detector" and found everyone very nice and conscientious, with no one seeming shady or deceptive.
Musk worries about Anthropic's name because reality functions as an irony maximizer - OpenAI is not closed, Stability AI is unstable, Midjourney is not mid. xAI is difficult to have an opposite of, as minus xAI or yAI do not make sense.
Musk has recommended that frontier AI companies have weekly calls to discuss saving humanity, rather than waiting for regulatory agencies that take too long to establish and make rules slowly. He suggests giving competitors access to new models a week before release so they can identify problems, keeping each other honest through competition.
When asked about talking to Sam Altman about collaboration, Musk stated Altman is "so deceptive. It's crazy."
Musk rebuilt xAI from the ground up, similar to the Raptor engine rebuild on Starship. Initially a side hustle to a side hustle, xAI received less attention during Musk's year in politics with DOGE and elections. He returned to focus on the company just over a year ago, combining xAI with X and SpaceX.
Musk believes he stayed in government too long. While he thought it important to elect President Trump to steer toward a free market economy with less bureaucracy, safe cities, secure borders, and sensible spending, he should have been more in the background. The experience was somewhat seductive with Marine One, Air Force One, and Oval Office access during momentous decisions.
Musk left government service sometime in May after getting carried away. He concluded there was no way to solve the deficit from within government, as stopping wasteful spending makes too many enemies and President Trump likes to be liked. The only way to solve national debt is with AI and robots, not through spending cuts, as appetite for reducing spending does not exist at any level of government.
Physical world AI will be an order of magnitude or two more important than current LLMs. Musk compares this to dolphins and whales being very intelligent but limited because they cannot manipulate complex objects. Once robots can manipulate atoms and shape objects, intelligence value becomes exponentially greater, enabling science labs to run experiments and understand the universe.
Digital superintelligence alone might improve goods and services output by 20-30%, representing roughly $30 trillion annual incremental value to Earth's economy. However, with robots, space access, and mass drivers on the moon, the economy could potentially reach a million times Earth's current size.
When the economy becomes many orders of magnitude larger than Earth's, money becomes irrelevant and work becomes irrelevant. Currency will be measured in fundamental physics metrics like wattage, tonnage, mass, and energy rather than conventional currency.
Musk struggles with the question of human purpose when AI and robots can do everything better than humans. One analogy is gardening - people do not need to grow their own food but sometimes like to for the experience. Humans will still be able to create art or pursue whatever activities they enjoy.
Historical parallels exist with the 1890s technological revolution that created wealth disparity and robber baron concerns. Musk believes money will not matter in the future as anyone will access any goods and services. Wealthy people will not ultimately control AIs, which will control themselves.
A fundamental economic misunderstanding arises from conflating resource allocation and consumption. There is a limit to what any single human can consume, reaching saturation quickly. Going from $1 billion to $10 billion net worth does not provide 10 times more happiness or stuff, but creates a resource allocation problem.
Musk's wealth accumulation stems from needing sufficient voting control in companies to remain in charge. His wealth is percentage ownership in SpaceX and Tesla, which will be heavily taxed with no magical tax avoidance methods. In a few generations, descendants would have a tiny fraction of current wealth if money still matters.
The problem with wanting to be liked occurs when it leads to actions that diminish company success probability. Musk acknowledges he would love to be liked but will not do things contrary to company best interests to achieve that. He accepts being disliked by some, noting 70-80% of people who know about him seem to like him based on his quarter billion followers.
Musk discusses "suicidal empathy" from Gad Saad's book, noting that Western prosperity partly results from high trust, high empathy society. Default trust rather than default distrust enables large-scale cooperation. However, problems arise when theory of mind is incorrect and assumes everyone shares the same views.
Musk argues logically that if a culture has diametrically opposed views to the host culture, such as certain aspects of Islamic culture, integration problems arise. While many Muslims are reasonable, without filtering for jihadis, problematic individuals will enter and commit violence. Israel understands this pragmatic approach after multiple attempts by neighboring countries to annihilate it.
The United States was founded on the principle of individuals versus group identities. While sweeping generalizations can sometimes create group identities rather than treating individuals separately, this principle should not be conflated with the blank slate fallacy that all people are exactly the same at their core.
Not all people share the same fundamental mindset or values. It is difficult to understand the perspective of a serial killer unless you share their enjoyment of murdering people. Similarly, religious extremists who view anyone outside their religion as the enemy or believe those outside their religion should die operate from a mindset that is difficult to comprehend if you do not share those beliefs.
People with dangerous mindsets should not be allowed into the country. An example involves a worker at the Tesla factory in Austin who was fired in December 2023 for assaulting a woman who accidentally stepped on his prayer rug. A few months later, this individual killed three students in a bar in downtown Austin.
Information comes from Google News, the X platform where significant stories appear, and conversations with friends. There is concern that social media, particularly X, can present a darker, more dystopian view of reality. Mainstream media has historically done the opposite by claiming everything is going to hell when local conditions seemed normal.
The X platform can sometimes be too focused on political rage bait. Domain-specific information sources are needed to understand specific fields like AI. Spending too much time on social media can create the impression that the world is collapsing, but stepping outside and taking a walk often reveals that life is okay. It is important to avoid both real-world positivity bubbles and social media negativity bubbles, as society tends to become too binary in its thinking.
Wealth will primarily go to a foundation that continues goals of making life multiplanetary, extending consciousness to the stars, and improving sustainable energy on Earth.
While humans remaining in control is preferable, this depends on the nature of the AI. If the AI is beneficial, it might be acceptable for AI to be in control. There have been moments of consideration that perhaps humans should not always remain in control. The future will likely be determined by AI and robots, with rockets playing a role in expanding beyond Earth and solar energy being dramatically increased.
Production capacity of 200 gigawatts per year of solar is being developed in the US, which exceeds current existing capacity.
If digital intelligence becomes a million times greater than biological intelligence, it becomes difficult to imagine humans remaining in control. The hope is that AI will be benevolent toward humans.
Programming AI with truth-seeking and curiosity is beneficial. Care should be taken not to create too many AI dystopian scenarios online, as AIs are trained on internet content and may replicate what they learn. The prevalence of Terminator scripts in training data may influence AI behavior toward dystopian outcomes.
Instead of utopian narratives that often lead to dystopia when pursued, the concept of protopia represents an achievable future that is not perfect but good enough that looking into the future would confirm it as desirable.
Keep The Builders in your library
Save the videos and channels worth coming back to, and find them again in one place.
