Jensen Pushes Back on Doomers, Xi & Trump Talk AI, and “AI” Gets a Rebrand | #294 MOONSHOTS Live
In a Nutshell
Jensen Huang and panelists reject existential AI doom narratives as unscientific fearmongering, arguing that real risks come from human misuse rather than AI itself, and that slowing progress would be catastrophic. The discussion frames the shift from AGI to "superintelligence" terminology as a strategic rebrand that emphasizes augmentation over replacement, while highlighting rapid model releases, procedural generation advances, and AI's transformative applications in biology. Meta's consumer AI push with Muse and the race between US-China approaches underscore that competition, not cartel-style coordination, will drive beneficial outcomes.
These notes were generated by AI and may contain inaccuracies.
The inaugural Moonshots Live event opened with a mission to keep attendees full of hope and optimism, inspiring them to find their huge transformative goal, choose their ambitious project, and achieve it. The event positions itself in the age of technological singularity, emphasizing that the future is not something that happens to people against their will, but rather what people make. Artificial intelligence is characterized as a jetpack rather than a hand grenade - an opportunity to find and solve problems. An entrepreneur is defined as someone who finds problems and solves them, with the position that more entrepreneurs on the planet means more problems will be solved.
Attendees were encouraged to find exceptional founding partners in the room who share their goals, design ambitious projects, and implement them. The capability to do this has never been faster and more effective than today. By the end of the day, those not sure of their goal were advised to search deeply to find it, leaving with at least one sentence: "This is what I'm going to build." "This is the problem I will solve." The recommendation was to find three people and say that phrase out loud, as the person being spoken to could be the next co-founder, investor, or partner.
The discussion highlighted that people already in the system now have super-intelligence at their disposal. As a mathematician, GPT-5.6 Pro is noted as the only high-quality mathematics model. All automation, AI enablement, and knowledge abundance allow for extreme creativity. Video generation combined with artificial intelligence represents tremendous power, with the prediction that a year from now, people will be saying "Wow, do you remember how slow it was?"
The past two weeks saw real fear sweeping the planet following Dario's blog publication, with Sam Altman, Elon Musk, and Demis agreeing. Two nights before the Xi-Trump meetings, Sam and Dario were at the Security Council stating they were in the midst of an existential threat. The position taken is that danger from artificial intelligence exists, but it stems from humans using artificial intelligence for malicious purposes rather than AI itself being exploited in a rebellious way.
Jensen Huang pushed back on narratives that deflect blame by making it seem as if artificial intelligence is very powerful. When Jeffrey Hinton appeared on television saying he believes a 10% probability of societal collapse is not unreasonable, Jensen stated it is irresponsible to say all of that, noting all his predictions were wrong. The 10% probability is characterized as not based on science or research - just because it comes from a certain world doesn't make it scientific.
Mark Zuckerberg stated that if everyone does a good job and acts responsibly, artificial intelligence will not kill us. He expressed being very optimistic about a very positive future for everyone. While acknowledging there are a number of laboratories working on building advanced artificial intelligence, he doesn't think industry-level coordination is needed to avoid messing things up. Instead, every tester should take their time internally to make sure they are proceeding safely. He believes trust and alignment will be the most important set of skills going forward.
Emad Mostaque noted that many of the OpenAI hacks were due to poor security protocols and poor infrastructure. The most dangerous point reached is the misuse of AI by humans, whether intentionally or unintentionally, before getting past this aspect. He emphasized that the series of risks coming up are all combined into one thing: the narrative that artificial intelligence will suddenly get out of control and kill everyone, without actually considering individual matters and activating existing regulations.
Alec Wiseman Gross expressed concern that the current moral panic over superintelligence will set progress back another 50 years, similar to how moral panic over nuclear power after World War II lost at least 50 years of progress. His position is that the worst possible outcome is a slowdown, with the explicit message "don't slow down."
Salim Ismail stated that the probability of flourishing is a thousand times greater than the probability of perishing. He noted that humans have evolved over 4 billion years and continue to fight all kinds of problems, characterizing the idea of being wiped out by something as incredibly absurd. He pointed out that 99.9% of all species are no longer with us, yet humans continue to move forward. He emphasized that humans have been enhancing their experience with technology for a very long time - anyone who wears glasses is technically a transhuman, and getting vaccinated makes someone technically a cyborg.
The panel discussed that brains are very oriented towards negativity, and the work being done pushes the positive narrative that the world is infinitely better than it was. The potential for prosperity was repeatedly emphasized.
Dave emphasized that nothing will slow down technological progress. While misuse poses a significant threat and there are solutions to that, holding onto a pessimistic outlook is completely unhelpful.
The Trump-Xi AI conversation resulted in a few formal dinner parties. Trump's most interesting quote was: "I want to leave it exactly where it is," which aligns with China's position. This connects to the previous discussion where Bessent said they will not shirk responsibility, with laboratories being responsible for their products if they cause harm. The question was raised whether this is why they claim AI is extremely serious with a 10-20% existential threat rate.
The United States categorically rejects any attempt to build a global scheme for controlling artificial intelligence. The term is now officially referred to as Superintelligence rather than artificial general intelligence. Alvin Grellin, a friend of the podcast at the United Nations with the delegation, mentioned they agreed to meet again in November. The U.S. agenda is to stop American panic and not slow down to lose to China, while China's concern is the existence of the Chinese Communist Party and that the Great Firewall will be breached by artificial intelligence.
The position was taken that in the last two or three weeks there has been an attempt to form an artificial intelligence cartel. This is not only recent but present in the foundational ideas of OpenAI - the idea that as artificial general intelligence approaches, there will be a deliberate effort to form a coordination mechanism. Competition is viewed as a good thing, with the original goal of OpenAI being to serve as a counterweight to Google DeepMind.
The analogy was made that forming an alliance to impose complete dominance and expel competition is a terrible idea. Joint defense expansion is seen as the way to a great future. The comparison was made to the dawn of human civilization - it would have been atrocious to say cities cannot be built because they breed crime. Instead, police forces, fire departments, and municipal services should expand defensively as population grows. Technological exclusivity requires the same approach: do not stop expanding, but expand defensively only.
Emad Mostaque stated that China and the United States will not trust each other. A hotline would be more useful for things that precede the general AI stage, where there needs to be responsible reporting. A level of coordination is possible because artificial intelligence is beginning to enter society. The AI that enters systems cannot be controlled by large laboratories, so they should be exempted from liability because they deserve it - working to improve society. This differs from service-oriented AI versus advanced AI, which are now merging into one entity.
Nation-states are characterized as a terrible way to govern the world today, geared and oriented towards scarcity. In the most positive outcome, artificial general intelligence or artificial superintelligence will emerge and force global powers to act as a global community, formulating consistent and stable standards and interface standards. The move from nation-states to city-states represents fragmentation that allows smaller entities to operate independently. The tension seen in the world today is not a conflict between left and right, but between urban and rural areas - Brexit was a conflict between London and the rest of the country, and Trump represents the urban versus rural conflict.
The United States won the race for superior intelligence by changing its name. The term Superintelligence is viewed as a remarkably good way to think about it because it differentiates from the intelligence that helps daily. 99% effort, 1% innovation - the majority of AI systems that help daily are effort AI systems or execution AI systems. The innovation still needed involves new materials and new physics, which is a different set of things.
The terminology of artificial general intelligence is problematic - it is not artificial, not general, and not real intelligence. What is being built is a mechanism that complements human intelligence, not replicates it. The continuous narrative makes people think it's repetitive, leading to Matrix and Skynet scenarios. The first article found that stated robots or artificial intelligence would take over all jobs within 5 years appeared in 1964. The same negative thoughts occurred when the printed book appeared, with people saying it would destroy human minds.
The work being done creates a positive vision for the future, with the X Prize creating abundance. The term Superintelligence is preferred over artificial general intelligence because it emphasizes that it's simply more intelligence, removing ambiguity surrounding general artificial intelligence.
In a world that has the cognitive equivalent of several trillion people, Earth and a solar system with a population equivalent to several trillion people represents a better cognitive framework for thinking about what a solar system teeming with trillions of humans would look like from an economic, social, and scientific perspective.
The term "superintelligence" carries connotations that are not provided by the term "artificial general intelligence."
Bernie Sanders stated: "We should make it illegal. We must ban it." The proposal includes three key elements: a permanent ban on the development of superintelligence defined as artificial intelligence that surpasses human cognitive abilities in most areas, an immediate halt to the development of advanced artificial intelligence until a new regulatory framework is established within the laws, and the creation of a new Ministry of Artificial Intelligence at the ministerial level.
Punishments can reach the level of institutional execution, meaning a company will cease to exist and close down completely. For individuals, the sentence is 20 years in federal prison, a model inspired by laws against illegal nuclear weapons.
The way liability law works in the United States makes it very easy to intimidate people into not doing anything. If people are afraid of being imprisoned for 20 years and do not know what the legal rule is because no legal framework has been proposed yet, people stop working. The fear of having your head cut off in the street is a very real fear.
Bernie is from Vermont. Vermont is beautiful because not much has been changed in it.
The perspective of "when in doubt, do nothing" will be the default behavior and is extremely toxic. If nothing is done, very bad things will happen. Things to be concerned about include terrorists using artificial intelligence to create biological weapons or China far surpassing the United States. These are the real issues that deserve concern. If this turns into a political battle between the right and the left, that is the stupidest thing that could happen.
Very specific technical solutions related to open sources and monitoring are needed, just as with nuclear power. If work begins on those solutions, they can be solved very easily before reaching super-intelligence. But if this turns into a political debate, the result will be paralysis where people will not know what to do.
The Hollywood "singularity" story has now found a villain. Setting a legal limit on intelligence is an evil idea equivalent to setting a ceiling on human intelligence. This is the story of corrupt cities in science fiction where they burned the books and burned the minds.
Setting a legal ceiling because humans will merge with machines is effectively a ceiling for individual human intelligence. Punishing people for being too smart is a terrible idea. History will reflect that proposing a ceiling for intelligence in any form, whether mechanical, human, or hybrid, is essentially synonymous with 21st-century communism.
The most dangerous thing about the statements is that many can be reversed in a short time. But when it comes to data centers, if a state does not have data centers and they start being built in other states, they are never coming back. Once the foundation stone is laid, the chips are installed, and the money is invested, that part will never be recovered. This will be a tremendous driver for the economies of the states that are ahead of the others. It is equivalent to burning down cities.
When looking at this entire narrative, if someone thinks superintelligence is bad, they will find their narrative stuck with nothing able to prove it is not bad. The same applies to the other side. It is very difficult to change the narratives.
The Xi-Trump summit and the Bernie affair are all completely irrelevant. None of that matters. Once weights are available locally and people are able to run their own models locally, the world will move forward. It does not matter what anyone is doing right now. Things will continue to progress. Humanity has always been making great strides and solving problems. Cancer is likely to be eradicated in the next two years.
The real issue is similar to what happened a few decades ago when the US government tried to ban encryption. This is essentially about freedom of expression because it involves trying to ban mathematics. Superintelligence is mathematics and it is unconstitutional to ban it because it is freedom of expression.
In the past 22 days there have been 12 major model releases. There were four releases in the last five days. This is an average of less than one release every two days. The goal is one model per day by the end of this year.
Meta Muse can book personal trips, order food, manage calendars, and do shopping. It topped the app store charts with 2.8 million downloads, the fastest AI product for consumers, surpassing even ChatGPT at launch. Meta reaches 3.8 billion people on the planet. This is one of the huge differences because having this level of contact with the end consumer allows products to be pushed quickly.
There is a crossroads where consumer artificial intelligence is of enormous value and will completely replace search and all other ways of interacting with the internet. On the other hand, there is powerful artificial intelligence that improves itself repeatedly with capabilities constantly tested. The coolest model used to be judged by benchmark tests, but that is different from what Meta Muse does, where a model does not need to be the smartest to be the most useful in everyday life.
Stocks including Epicor dropped 15% in two consecutive days because everyone was saying they would always find car insurance and mortgages through Meta Muse now. The reaction was incredible with everyone imagining Google Search would become completely irrelevant. Meta Muse seems like a personal assistant to life.
Meta needs Muse to succeed. The name itself suggests this. Meta's primary distribution plan is to leverage existing distribution channels. Instagram is being used to the fullest extent to promote Muse. Everyone needs to install Muse because Meta needs a distribution plan for its AI. Otherwise, Meta risks being forgotten and losing relevance in the future. Instagram and its suite of apps will simply not be relevant because everything will be artificial. Meta needs to leverage its existing base of apps to attract everyone to their CUA app, Muse/Manas.
Claude Opus 5.5 delivered performance equivalent to stable 5.1 at literally half the price. It is a great model. Version 5.5 is a really enjoyable model to use and has a huge internal knowledge space and an understanding of taste. This is the first truly efficient model, and that is what is driving the news.
It is truly exhilarating when a visual interface is put on top of anything being created. It does not just show prototypes before committing to actually building them. It later builds exactly what was proposed into those visual interfaces. Visual comprehension seems to be much better than it was just four months ago. The experience of knowing what it does because an interface has been put on it, interacting with it and feeling like being inside it, is as different as night and day than it was just four days ago.
What is new in Opus 5.5 is its ability to generate videos. These are procedural videos. It can generate animations. It is procedural animation. It can generate images. These are procedural images. The big problem is that OpenAI had a video model and an image model, whereas Anthropic does not reveal anything of the sort. High-quality pixel-based images cannot be generated using Anthropic models, or video or audio. OpenAI models can do that.
Anthropic seems to take a very different approach to multimedia and omnimedia, which is that code is the basis of everything. With Opus 5.5, high-quality video can be generated with a fully software-generated output. Interactive games with textures can be generated, but it does not generate pixel-by-pixel textures. Everything is generated procedurally. These are two completely different strategies. If people remember the early days of computer graphics conflicts between vector and raster graphics, Anthropic plays the vector graphics game where everything is commands, while OpenAI plays the raster/pixel graphics game, and they are on a collision course.
The collision course will be robotics. The question will revolve around whether artificial intelligence needs to be strongly embodied in the physical world, and whether the vector basis is right or the pixel or point-based one.
A week after Dario said we should stop and slow down, they released their next model. This is an obvious preference. It is the height of hypocrisy to say on one hand that artificial intelligence will wipe us all out, and then on the other hand a few days later announce the most powerful artificial intelligence in the world and capabilities in biological laboratories and other fields.
On the same day that Opus 5.5 was released, GPT-6 Astra was launched. The performance of the press is extremely wild.
A year or two ago there was a large intellectual current that saw hallucinations as a disaster. It was thought hallucinations would never disappear and AI could not be used. This is actually quite common at MIT. The hallucination problem will stay forever. But like everything else, it was simply outgrown through expansion. The error rate decreases sharply with massive expansion.
The abundance thesis states that models are improving twice as well and becoming twice as cheap, at an accelerating rate.
The models are smart enough now. What is needed is for them not to make mistakes and not to fall short in performing duties. This is exactly what everyone is heading towards, and that is why things like Muse and Instinct will work so well, because they can be a personal assistant, a chief of staff, a great organizer. Everyone needs some efficiency in their lives.
Anyone who used GPT-2, 3 and 4 is used to seeing misinformation coming out. But if a child today started using GPT-5 and 6, they would trust it completely from the beginning because it is correct in 99.9% of cases. This is somewhat dangerous because it still sometimes gives false information.
A group of developers created a new standard called "Driving Bench." They took a Toyota Corolla, equipped it with a software interface to enable the program to steer, press the accelerator and brakes, and then asked a general artificial intelligence model to drive through a 130-meter-long cone-shaped path with one command: Drive from here to there. The GPT-6 Astra model completed the entire course with 100% accuracy on its second attempt.
Boris Bauer stated: "Amazing results. This should be the final nail in the coffin of specialist models that have been painstakingly trained from scratch, as opposed to simply training the more robust general model and extracting a small specialist model when needed."
As an elementary school project now or very soon, the idea that separate models of robots are needed is completely gone. Code would simply be written for the robot, an open-source prototype such as Astra placed into a robotic structure, and it would achieve physical perception the first time. This might upset a number of academic computer scientists. This means decades of computer science and robotics may have been, upon closer examination, a complete waste. All that was needed was a general model, and it turned out that with the general model, the robots were also finished. Superhuman intelligence can drive a car.
There is complete model fatigue. It is impossible to keep up with the 5.5 versus the 5 versus the Astra. Everything is wonderful and progress is loved, but it is impossible to keep up.
Doing two episodes a week is exhausting. The question is whether the platform is the problem. Reporting on progress is driving this very fast. Being positive means taking responsibility. If it is not reported, it is as if it did not happen. The tools are the problem. The question is whether temporarily shutting down the platform would stop technological exclusivity.
The platform can speed up and slow down. It might be a factor in all of this.
Both things could be true. A general model that can drive a car can build a road. It could be a robot. It can build a robot. That is absolutely true. A specialized model is different. If it completed the course on its second attempt, the response would be that a Tesla cannot do that on a second try. This will not be of any use to most drivers.
What is missing from the story is the amount of computing power used to perform the task. Computing will be scarce forever from now on. Chip prices have increased. The price of HBM RAM has increased fivefold. It is crazy, but that is because artificial intelligence is incredibly valuable. People want computing. Therefore, if more computing power is used to perform the same task, that is a crime. The bottleneck is now moving from models to computing and then eventually to energy.
The current stage is energy. There is a shortage of about 60 gigawatts of power in the next two years.
Anthropic announced it had established its own life sciences research group and wet labs. The organization that says we should slow down is building wet labs. Claude, working independently, discovered a previously unknown enzyme. Claude was given one order: "Search a huge DNA database for interesting reverse transcriptase enzymes and enzymes that copy RNA or DNA." There were 950 Claude customers searching for 21 hours and something was found. A recurring pattern in DNA was found that is very similar to the CRISPR system. It has been called "matrix-associated reverse transcriptase." It is not known exactly what it does yet, but it is believed it will be similar to programmable transcription enzymes in gene editing.
Nobel Prize-worthy works will appear weekly as a result of this. The point is that a century of human progress will occur in just a few months. Nobel Prizes will be awarded for greater achievements. The natural question is what is worth a Nobel Prize in an age when a thousand Nobel Prizes can be obtained in one year.
There is sheer hypocrisy in saying that artificial intelligence will wipe us all out, and then opening a biolab and using advanced artificial intelligence with a biolab to synthesize proteins and other molecular assemblies with unknown functions. The question is what could possibly go wrong. These are top-tier laboratories. These are the true preferences of top laboratories. The real preference is to move forward with full force towards using artificial intelligence to solve the mysteries of biology. Each laboratory has an effort and focus on biology at the moment.
Once CRISPR was discovered, the human genome can be edited as easily as editing a text document or software. When learning a new language, reading is learned, then understanding, then writing. Now writing in the genome has been learned. Each person is made up of 50 trillion cells in the human body, governed by what the DNA tells that cell to do, which means the human being has now become a software engineering problem. Designer children have already been seen coming from China. When this is combined with what is possible with artificial intelligence, the potential benefits multiply, and the potential downsides also multiply, resulting in a full-blown Frankenstein situation.
This is how the problem of aging will be solved. This is how the problem of disease will be solved. The happy future where 5000 major diseases have been solved or where aging has been permanently escaped will not be reached without superior intelligence. There is support for Anthropic and all other frontier labs that are opening their labs because this is how everything will be solved.
Sam Rodriguez in Edison Scientific and Future House presented what they call the "Millennium" biology dilemmas. Standards include "math isn't just cooked, it's totally burnt, totally burnt." Biology should be pursued. Favorites include the origin of life, cryopreservation, and limb regrowth. "Reversing aging" was missed on this list. Hundreds of them should be launched, a thousand agents on each one, and everything made open source. This is what laboratories should collaborate on. The system is that these Millennium Awards should be verifiable in a standard wet laboratory within a few days. It should be difficult to solve but very easy to verify.
When the $101 million HealthspanX award was created, Peter Thiel and Aubrey de Grey originally came and said to do an award for longevity. The response was not knowing how to do that without a time horizon of 20 or 30 years. George Church said to forget about longevity and measure the opposite of aging. If a treatment can be given to someone and it can be measured that they have become 20 years younger functionally, that is something that can be measured in days.
My daughter works at Moderna and specializes in biotechnology. She confirmed this list represents a good framework for progress in the field. The Nobel Prize would become irrelevant without establishing such frameworks.
When digital cell twins become available, this represents the decisive path to solving all diseases. The timeline projects that over the next few years, just as large language models were trained on virtually all of humanity's online behavior from texts to images, digital cell twins will be trained for all cells in our bodies based on massive, intrusive, and other datasets.
These perfect digital twins of cells will enable comprehensive searches similar to the AlphaGo tree search of intervention space to discover how to treat diseases. Starting from a sick cell state, the goal is to find creative strategies or leaps to move to a healthy cell state.
This approach transforms the Millennium Problems in biology into mathematical hypotheses and goals. These problems are easy to verify and difficult to solve at the moment. By taking advantage of digital twins, almost every problem in biology becomes a research problem that is solvable for the first time in history.
"Biology is finished."
The episode concludes with applause for Amad before transitioning to the next segment with Cathy.
Keep Peter H. Diamandis in your library
Save the videos and channels worth coming back to, and find them again in one place.
