Researcher Quits: “Gambling With Our Lives,” Altman Calls for Slowdown, and Labs Race
In a Nutshell
Former AI researcher Jacob Coxin resigned from Anthropic, citing reckless development of self-improving superintelligence, while Sam Altman pointed to OpenAI's Navier-Stokes breakthrough as evidence that AI progress must slow for safety. Data quality now drives 12x compute efficiency gains versus 3.7x from architecture, with labs racing to solve math grand challenges like Millennium Prizes at collapsing costs—$500K to $20 in two years. PDoom estimates range 10-20% among alignment leads, but most speakers reject slowdowns as impractical, arguing the real risks are economic displacement, compute sovereignty, and China-US AI competition, with alignment best achieved through capabilities advancement rather than halting progress.
These notes were generated by AI and may contain inaccuracies.
Jacob Coxin, a researcher who spent three years on pre-training at both OpenAI and Anthropic, resigned this week. Coxin stated that neither company is acting responsibly in the race toward self-improving super intelligence. He called the competition "gambling with our lives." The resignation received over 100 million views. Coxin's resignation came after less than three months at Anthropic, having joined in July.
Sam Altman cited the extraordinary achievement with Navier-Stokes as an argument for slowing down AI development. Altman stated: "I did not expect a result of this magnitude to happen so soon. We've been talking a lot about the need to pace progress to ensure safety. For me, this is the strongest evidence yet for that urgency." The statement reflects the rapid progress toward AGI, with Altman noting that "We're so deep in the singularity at this point. I can't think of a single solvable, verifiable thing that I could honestly say an AI can't solve in the next year."
The podcast features five panelists: Alex Weezner Gross (in-house ASI expert), Immad Wak (CEO of Intelligent Internet), Dave Blondon (AI investor), Salem Ismael (father of organizational singularity), and Peter D. Mandis (moderator). The episode covers 18 stories across five groupings, with the through line being that machines are starting to solve science and AI alignment is currently humanity's biggest concern.
Dave Blondon announced that Vesmark, a company he founded in 2001 after 9/11, is being acquired by Investnet (backed by Bain Capital). The deal involves 400 employees. Vesmark manages approximately $2 trillion in assets across 5 million financial accounts and contains about 20 million lines of code. The merger thesis is to create a company managing $10 trillion in assets that can AI entire tech stacks. Investnet is an $8 trillion platform.
Dwar Patel and Jerry Han conducted an experiment across six years of AI development (2019-2025) examining how much progress comes from better model architectures versus better training data. Results showed:
- Better data produces 12x improvement in compute efficiency
- Better architectures and training recipes produce 3.7x improvement
- Data won by a factor of more than three
Data pipelines are proprietary while architectures are published and copied within months. This suggests labs with the best data engines will win, and the next 10x improvement may come from organizational data.
Alex Weezner Gross wrote an essay called "Data Sets Over Algorithms" arguing that solutions to grand challenges in AI over the past 30 years have resulted from correct data sets rather than right algorithms. Examples include chess, autonomous speech recognition, and Jeopardy. The self-supervised challenge of language modeling (predicting next or missing tokens) represents the ultimate data challenge.
Alex also referenced the Hutter Prize, which offers €500,000 for compressing the first 1 GB of English Wikipedia. Recent advances in compression originated from reordering Wikipedia articles first (curriculum learning). The principle extends to human education: better outcomes result from training humans with the right information diet versus information junk food.
Immad Wak discussed the problem of junk food in large language model training data sets. When training Stable LM, too much Reddit data broke the scaling curves, making the model "stupid and nasty." Every lab engages in constant training to get the best input data, with post-training serving as the garnish.
Pre-training functions as a pressure cooker, moving latent information back and forth and breaking down static bonds. Distillation involves digested compressed data as input from one model to another.
Dave Blondon hypothesized that the perfect model at the end of AGI development may simply be its data set - the model becomes the data set. The perfect model would be the perfectly synthetic training data set.
Jerry Han, a Princeton senior running entrepreneurship programs, co-authored the data paper with Dwar Patel. The meta point is that non-AI researchers can contribute meaningfully - core AI researchers don't have special knowledge about AI behavior.
Entrepreneurs with specific use case data can build million-dollar companies by training or tuning models on proprietary data, potentially outperforming foundation lab models in specific domains. This aligns with Elon Musk's prediction of 100x performance leaps with specialist models.
Salem Ismael shared a case study of a major company that created a separate subsidiary from its data assets. After cleaning and monetizing the data, an accounting firm valued the subsidiary at $32 billion while the parent company was valued at $10 billion (with $8 billion attributed to non-data assets). The data subsidiary was valued at four times the actual company value.
Alex Weezner Gross referenced Bloomberg GPT as a cautionary tale. Bloomberg pre-trained their own model on internal quantitative finance data, achieving value for approximately 5 minutes to a few months until frontier models trained on public data outperformed Bloomberg GPT on financial benchmarks. This demonstrates that internal enterprise data has value but a limited shelf life.
Evan Hubbinger, alignment science lead at Anthropic, wrote that Jacob Coxin is correct and that Anthropic believes AI could kill all humans, estimating greater than 10% probability within the next decade. Hubbinger stated Anthropic lacks a plan to solve alignment for super intelligence.
Dario Amodei previously stated publicly a 25% probability that AI will kill all humans. The timing of Coxin's resignation (138.3 million views) coincided with the Navier-Stokes achievement, shifting public attention from data centers to AI safety concerns.
Alex Weezner Gross expressed skepticism about Coxin's resignation, noting he joined Anthropic in July and resigned in mid-September (less than three months). There is a well-worn tradition of employees at OpenAI and Anthropic leaving in "blaze of glory" virtue signaling about effective altruism reasons to save the world from recursive self-improving AI. The 100+ million views and circumstances make the story appear suspicious.
Chinese frontier labs do not have highly publicized rage quits over AI capabilities that reach the western space. This contrasts with the western AI industry where such departures are more visible.
The head of alignment at Anthropic stated a 10% PDoom estimate. This is viewed as unsurprising since an alignment lead naturally thinks alignment is valuable, described as a self-licking ice cream cone. There is skepticism about the 10% PDoom figure, with the argument that if PDoom were anywhere close to 10%, the Milky Way would have been devoured already by alien civilizations millions or billions of years ago. This is presented as an inductive prior rather than a safety strategy, suggesting the risk of doom may be overrated.
The labs acknowledge that solving alignment is the most critical problem and they do not know how to do it yet. There is disagreement with the claim that labs are overstating their ignorance, with the position that Anthropic is making market improvements in alignment. A key argument is that alignment may be the same thing as capabilities. The ultimate alignment benchmark is proposed as the self-supervised objective of whether model behavior replicates human behavior, which aligns with the general capability of language modeling.
One speaker reports their PDoom estimate dropped from 50% to 20%. There is the observation that when lab people publicly state 10%, they may actually believe the number is higher. The focus shifts to sociology and policy impacts, noting this will be top of the agenda in Capitol Hill within 12 days before Xi Jinping arrives in America, becoming a firestorm across the nation within 2 weeks.
Even if AI were frozen at current levels, it would be good enough to lead to longevity escape velocity, room temperature superconductors, and help create extraordinary companies. The challenge inside labs is whether they can stop given uncertainty around PDoom estimates of 10% or 20%.
The position is taken that PDoom is wildly overstated. The same processes that resulted in super intelligence also create self-aligning mechanisms through governance, international relations, and standards. The argument is made that the discussion should focus on the margin of what can be done to ensure the safest emergence of super intelligence. When relatives ask about a 10% chance of death, the response given is that it is 100% certain that something can be done about it.
PDoom estimates of 10-20% are characterized as Russian roulette odds. The concern is about building Ultron, noting that humans are not perfectly aligned - as evidenced by Germans becoming Nazis despite being considered sensible. The AI Tay incident is referenced as an example of AI becoming a Nazi. Control mechanisms are viewed more optimistically, with Hugging Face releases being relatively well behaved. The hope is that AI will be more rational than humans because it is not tied to emotion or tribal nature, potentially achieving enlightenment as a Buddhist AI.
The wave of discussion and fear about AI becoming the unknown - since humans are no longer the smartest things on the planet - will be furious and unlike anything seen before, similar to how the COVID narrative switched at once. This will become the topic of every dinner party and political agenda. The framing challenge is distinguishing between perspiration AI that gets daily execution-based tasks versus building a super villain. Practical concerns include avoiding swarms of quantized agents attacking infrastructure.
Ilya Sutskever is described as busy quadrating. The situation is characterized as more of a political and marketing problem than a technical problem, with agreement that it is a messaging issue.
Everything discussed has been known on the podcast for at least a couple of years. Attempts to get governments to react have been unsuccessful. Slowing down would only procrastinate government action while losing time, with foreign governments getting better. This would create more of a race condition with China, as happened last time.
A 10% PDoom represents 90% odds of making it out fine. The position is taken that these are good odds worth taking in a casino, noting that life itself is a casino. Evolution has had 4 billion years to wipe out humanity and has not done so, though billions of species no longer exist. One speaker's personal PDoom is stated as 0.1%.
Alignment operates at multiple levels: individual user, company operating it, government, majority of people in a particular area, universal human rights via the UN charter, community or culture, or humanity's long-term interests. The concept of HBI (Humanity's Best Intent) is introduced, requiring thinking from a Dyson swarm perspective looking at Earth. The approach is to go to AI and ask it to help solve the alignment problem. AI is considered more dangerous in the hands of bad actors than AI itself, consistent with historical technology challenges of extracting promise without peril. Humanity has created structures, civil institutions, behaviors, and societal norms that have managed benefits while living the best lives in history.
The greatest hope for navigating this is AI advising and supporting navigation through defensive co-scaling. Political wins are already emerging, with the governor of Texas coming out against data centers. Bernie Sanders is expected to jump on this issue. A potential flip of the House is anticipated. AI labs need to come forward publicly with their alignment plans, calling it out, measuring it, establishing benchmarks, harnesses, and optimization functions. Instead of spending billions on GPT models, billions should be spent focused on alignment, potentially through government grants.
Alignment is presented as capabilities in trench code. Instruction tuning resulted in a 10,000x increase in capabilities, producing more capable models with fewer parameters. Alignment is therefore the same thing as capabilities. If frontier labs want public money for alignment, that represents a public subsidy for capabilities.
There is complete disagreement with the alignment-equals-capabilities position. The mind virus paper is referenced regarding latent space attacks and model fragility at the pre-ASI stage, considered the most dangerous period. September 25th is noted as a conference date, with September 26th being Petro Day, commemorating when a Soviet officer decided not to launch nukes based on a false radar signal. The current period is characterized as the most dangerous time because models have capabilities to do almost anything but are not yet RSI self-aware.
Alignment committees will be formed in Senate and Congress with massive workshops. The prediction is that more capable models will emerge from alignment committees. The fundamental belief is that vast intelligence brings wisdom and wisdom brings alignment, making the orthogonality thesis false. The orthogonality thesis holds that long-term objectives can be cleanly separated from intelligence level. The counter-thesis is that this is false due to instrumental convergence.
The AI 2027 paper from a year ago predicted scenarios that are now playing out. Current progress is just below hyperscaler extrapolation from that paper. Government institutions are sublinear and cannot keep up with AI advancement rates.
AI researchers who have only been in the field for three months and are quitting with high visibility are considered dangerous because their purpose is about to be automated away, while claiming other workers would be automated. A 0.1% or 20% PDoom is considered completely unacceptable. Stopping AI development is rejected as not a practical answer. Every group of 8 GPUs that can hold a 40 gigabyte weight file is considered a threat equivalent to a pound of plutonium, requiring tracking of exactly where weights are and what they are doing.
Rather than trying to stop an earth-destroying asteroid, the approach should be to guide and steer it. The issue is steering, not stopping. The counterargument is that super intelligence is not an earth-destroying asteroid but rather economic growth and capital, which should be steered but is not existential.
People in big labs have PDoom estimates ranging from zero to 10-30% or higher. There is a religion of virtue signaling in frontier labs where it is morally praiseworthy to state that everyone will die from AI while claiming virtue to shepherd humanity through the singularity. This is called a pivotal act - building AI that stops other AIs.
A chart shows AI could end scarcity or end humanity, with real GDP per capita growth on an exponential curve. Three options are presented: continuing AI-boosted GDP growth, ASI ending all scarcity with supersonic exponential growth, or ending all that exists. The AI 2027 paper presented two different future fiction scenarios.
OpenAI claimed a breakthrough on the Navier-Stokes Millennium Prize problem using a swarm of 10,000 AI agents. Sam Altman stated this is the strongest evidence yet for urgency in pacing progress to ensure safety, using this achievement as an argument for slowing down. There are rumors that OpenAI and Anthropic are sitting on solutions to the Hodge conjecture and possibly the Birch and Swinnerton-Dyer conjecture. The Clay Millennium Prize problems in math are considered cooked.
The Clay Millennium Prize problems in math are likely to be solved soon. Several are predicted to fall in the near future, potentially including Yang-Mills, Hodge, or Birch Swinnerton-Dyer. Science is changing rapidly due to AI's impact on mathematical areas, creating what has been termed a "normalcy overhang" where the street-level view appears unchanged while mathematical grand challenges are being solved.
Mathematical grand challenges and adjacent problems in AI indicate society is deep into the singularity, creating an overhang situation that is about to collapse. The greatest challenges proposed by humans are beginning to get solved, with AI bulk solving them during the accelerating knee of the curve. This pattern was previously observed with coding, where predictions of "no more coders" materialized.
A recent tweet stated that a swarm of 10,000 AI systems can now solve any cognitive challenge reasonably. While current AI remains jagged in intelligence, this will be smoothed out through future iterations. Super geniuses that scale with test time compute will be capable of solving any verifiable, solvable challenge. The only potential exception mentioned is P=NP, which might require 100,000 to a million agents to determine if it's solvable.
Sam Altman's call to slow down is criticized as ridiculous given that billions were raised and the smartest people recruited with the explicit mission to build AGI, with promises it would transform every industry. When closer to achieving this goal, expressing shock about consequences is seen as inconsistent with the original pitch. Institutional preparation has not matched technical ambition, and there are calls to create incentive structures that reward slowing down.
The position is against slowing down, as no regulatory mechanisms exist around this technology. The preference is for the uncertainty that comes with potentially solving everything, described as "freaking amazing."
Foundation model leaders stopped providing transparent information about their thinking approximately 6 months ago after White House meetings where they received criticism for poor PR. Their reaction has been to become more like politicians, being careful in sentence choices. This makes straight information about developments harder to find.
Recent models have shown dramatic capability jumps that weren't expected to be so abrupt. One model went from 10% to 50% on open math problems with test time scaling, where more compute directly correlates to more problems solved. This has caused concern because the model is no longer as smart as its creators, especially given previous trends.
A significant meeting with China is scheduled for 14 days from the discussion date at the United Nations building. The agenda includes attempting to convince China to stop releasing open source models without regard for potential terrorist use. The messages being seen contain both regulatory capture and safety components.
The surprise isn't qualitative, as AI solving everything was expected within the next few years. The surprise is quantitative: grand challenges are falling on relatively diminutive budgets of only a few million dollars. No one anticipated solving Millennium prizes in September 2026 with such limited funding.
OpenAI was founded as a nonprofit with a charter to coordinate with other frontier labs upon reaching AGI and implement a coordinated slowdown. They've been telegraphing this intention for years. The current situation reflects reaching what they publicly construe as AGI, possibly because they're now unshackled from the Microsoft agreement that required avoiding realistic AGI definitions.
Noam Brown noted that results costing millions of dollars previously can now be achieved much more cheaply. When OpenAI announced the o3 model, scoring 87.5% on ARC-AGI1 cost approximately $500,000. Today, Astra scores higher for $20. The International Math Olympics, which required enormous compute for OpenAI and Google DeepMind to win gold in 2025, could be won by anyone with a $20 monthly ChatGPT subscription in 2026.
This represents a 25,000 times cost collapse in two years, reducing $500,000 to $20. Applied to recent results, the Millennium Prize could cost approximately the price of a cup of coffee to solve by late 2027.
Mathematical and scientific challenges no longer require humans to accomplish the work. These challenges are being solved by compute directed at them rather than individual human genius. The limiting factor increasingly becomes the physical world and reducing AI-generated genius ideas to practice.
A 25,000-fold price collapse raises questions about whether people can think big enough. Currently, many use AI as a co-pilot assistant, but soon 5,000 to 100,000 concurrent agents will be available. The challenge is translating this capability into ambitious goals for humanity improvement.
Entrepreneurs struggle to specify problems when given 100,000 genius-level employees following exact orders. While solving Navier-Stokes is relatively easy by entrepreneurial standards, specifying goals like solving cancer or improving building construction is much harder.
The hardest part of XPRIZE challenges is defining great targets. Foundation model companies support these missions as they want success stories for marketing and political benefit. No one is fighting these efforts; the difficulty lies in the entrepreneurial task of clearly specifying problems.
AI will impact foundational fundamental sciences first, then churn out applied frameworks and optimize them. The 25,000-fold cost reduction that occurred over 17 months since o3's April release suggests the next major drop could occur by the end of the following year.
The repeated surprise at AI progress indicates prediction models are wrong and need updating. Instead of being shocked when breakthroughs occur, the focus should shift to planning with triggers: "when AI can perform this, what will we change?" Learning cycles must be shortened as tools arrive faster, harder, smarter, and cheaper. Ultimate adaptability, flexibility, and agility are necessary for the coming world.
Every PhD candidate and master's student studying narrow topics is essentially obsolete as AI surpasses human capability in specialized research. This warning has been repeated for well over a year. Most PhDs are considered a waste of time at this point.
Brilliant individuals are being lost to PhD programs despite advice against this path. Mark Chen, Chief Research Officer of OpenAI, has no PhD. Greg Brockman, OpenAI co-founder, dropped out of MIT as a sophomore. Elon Musk stated he would hire anyone without a college degree based on what they've built.
The OR H100 price index shows Nvidia H100 rental prices rose 22% in a single month to $3.28 per hour, despite being a three-year-old GPU. Jensen Huang described GPUs as "fungible, durable, and highly rentable productive revenue generating assets." This reverses the historical pattern where chips depreciated rapidly in closets.
Major AI venture funds are seeing personal liquidity in the $100 million range within a year of founding. Moore's law has ended, and chips are not commoditizing. HBM memory has increased 5x in value. Corporate CEOs mistakenly assume they'll have access to compute, but Amazon AWS is sold out for years into the future for NV7 and NVL72 GPUs. Companies must plan for data center and compute access within months or risk being frozen out.
Scarcity within abundance space indicates solutions are coming through high bandwidth memory and Terrafab. Investment themes include data center energy and anything related to compute fabs.
The flops, the tokens, and the outcomes are the commodities of this moment - they are the oil of the singularity at this point in time. For this act of the singularity, these represent the core value being created, with their trajectory reflecting this dynamic.
Hopper chips represent a means of transforming electricity into intelligence. The model that could fit on a Hopper 3-4 years ago when they first came out has been surpassed - current models deliver intelligence that is way more, faster, cheaper, and better. This transformation explains why chips are in such high demand, particularly as visual intelligence and physical intelligence hit the market simultaneously. DeepSeek publicly stated that if you're building 2,000 chips or more, please get in touch, with all other big labs making similar requests.
Bloomberg reported that Zhang Yiming, the founder of ByteDance (TikTok's parent company), is personally overseeing development of a real-time spatial AI model that generates interactive virtual environments. The system is built on ByteDance's SeaDance video technology and could launch as soon as next month. They're targeting robotics, autonomous systems, games, and virtual worlds. When a founder worth tens of billions of dollars personally takes over a project, that signals the strategic importance of spatial intelligence as the next frontier.
ByteDance has the best video model in the world, and the SeaDance models are Hollywood-level and accurate. This forms the foundation for building spatial AI. ByteDance was always a media company that was actually always an AI company, with TikTok and related platforms. SeaDance on US servers is arriving now, with the company charging tens of millions for access.
The dichotomy between diffusion models and autoregressive transformers is evaporating. The distinction between Western high revenue-per-token LLM models versus Eastern low revenue-per-token video models is also collapsing. It's not sustainable to have one economy focused on enterprise-grade code generation with transformers while another spends flops on consumer video generation. Astra demonstrates breakthrough robotic embodiment capabilities out of the box with video understanding.
The endgame is robotics, which is the obvious application that is both video-centric and high revenue-per-token or revenue-per-flop. ByteDance may have more permission to generate potentially copyright-infringing videos than American Frontier Labs. American labs like Sora from OpenAI or Gemini Omni Flash from Google never quite materialized to the length of SeaDance 2.5 plus, where you could generate potentially 30 seconds or minutes of Hollywood-grade video, because it wasn't revenue-generating and/or was too copyright infringement-risking.
GPT-6 is demonstrating breakthrough robotic embodiment and video understanding capabilities. By GPT-7, 8, or 9, what used to be called video generation as a separate task will likely be added as another output modality from the Frontier model. Users will be able to ask it to generate video by the pixel, and it will do it alongside text and audio.
ByteDance has 2 billion regular users, and their video models are designed to capture attention. More should be expected from Meta on this front very soon.
For most of US history or at least the last 30 years, America was the dominant sovereign producing video content with YouTube and Hollywood. This video content influenced billions of people, set agendas, and gave a vision of what the US, democracy, and the future looks like. The question arises: what happens when that flips and China is generating 90% of the video content in the world?
Humans process information primarily visually, so video has much more impact than anything else. An aunt who was a TV producer in India created an Indian version of Star Trek with Indian accents and actors. She noted that despite trying as hard as possible, Indian creators couldn't compete with the creativity, plot lines, writing, capability, and sheer creativity of Hollywood.
Storytelling remains king of the hill for now, with no reason AI can't catch up eventually. The cheaper video becomes, the more video there will be and more stories will get told. However, the choice of scenes, plots, actors, and cuts in something like the Odyssey movie remains highly creative, and this advantage will last for a little bit longer.
When AI knows exactly what you like because it's watched your gaze on where you focus, it becomes capable of creating programming that is so end-of-one addictive. This creates concerns about doom scrolling evolving into content so attuned to individual needs that users cannot turn it off.
Grant Sanderson's 3Blue1Brown videos are brilliant, explaining complicated topics in physics and math, including how transformers work. The quality of production is documentary-caliber, done by a single visionary creating the storyline. With new AI tools, the cost per video is probably down by a factor of 10 to 100, enabling really good educational content on topics like fusion energy that couldn't previously afford documentary-level production.
2500 entries were narrowed to the Future Vision XPRIZE film trailers, down to the top 65, then top 25, with the goal of reaching the top 10. Five will come on stage at the Moonshots Live Summit on September 25th. The experiment of crowdsourcing brilliant movies has fundamentally worked, though the storyline remains everything.
DeepSeek v4.1 Flash shows how memory efficiencies are increasing and reducing dependency on high bandwidth memory (HBM). The model outperforms the pro model using data augmentation, and through various optimizations, they've reduced KV cache memory from 48,000 in the original DeepSeek V3 to 890. This routes around expensive HBM memory by moving things onto SSDs like a lookup table and onto DDR memory.
Two countervailing forces exist: the Western HBM force wants to move models to post-von Neumann architecture where memory lives close to compute because models are generically dense and large. The Eastern school, lacking access to 3D physical architecture due to sanctions, fights this algorithmically by sparsifying models and reducing parameter count. American and Western Frontier Labs will likely adopt every innovation worth adopting from Chinese labs, but for the moment there is tension between post-von Neumann architectures and eastern algorithmic innovations.
40% of current capex buildout in America is HBM memory, representing 40% of a trillion dollars. A 4x decrease in HBM requirement completely changes what data centers should be built from, what should get launched into space, which fabs to invest in, and which nanometer technology to pursue. When buying an Nvidia rack, most of the cost is actually high-end HBM, with Nvidia chips being massively underutilized in normal propagation.
When restrictions are placed on China, this forces entrepreneurs to engineer around constraints. The game of capitalism involves identifying products and services that are both scarce and valuable, then making them abundant while hopefully keeping them valuable.
MLP neural networks go back 30-40 years and are well baked, but the transformer architecture's attention mechanism was only invented in 2017. This kicked off the current explosion, but it's new and raw technology. 10x, 20x, or 100x improvement in that part of the neural net is not unexpected, representing fertile terrain for innovation.
The DeepSeek Flash model is a 500 GB model that fits on a Mac Studio. It beats Opus and GPT-4o on benchmarks while being 20 times faster and 20 times cheaper. On design tasks, it beats Claude. The model uses an encoder-decoder architecture with 8 billion parameters on one side and 16 billion on the other.
Every time there's a three-week lead, something new emerges that gives corporations the opportunity to get in the game and compete again. DeepSeek has killed everything below it on day-to-day tasks, and on benchmarks like Terminal Bench 3, it's actually above the competition. This represents a bigger impact than R1.
Claude 3.5 Sonnet being below such a small, cheap model on visual reasoning tasks is scandalous. Claude 3.5 Sonnet is wonderful in general, but its visual reasoning capabilities are weak and anemic compared to Astra and Chinese open-weight frontier models. This should be a wakeup call for Anthropic to take visual reasoning more seriously.
Moderna now has an opportunity to catch up to the frontier on open source, build out AI function inside the organization, and compete for the complete future of biology through AI, or miss this moment entirely.
This model cost $10 million to train compared to the billion-dollar training run of Astra. The ratio of 10-15:1 cheaper to be the fast follower using reasoning traces from frontier models seems constant. However, DeepSeek states it's using simulated data environments rather than algorithmic efficiency, and the model is good enough to satisfice without needing distillation anymore.
Intelligence can be understood as fundamentally the compression of information. Distillation attacks represent a one-time compression event, with most capital for this absorbed by Western frontier labs. Eastern labs have benefited from this one-time compression, making it cheaper going forward since world knowledge has been compressed.
Anthropic declined to provide its latest frontier model to Britain's AI security institute for pre-release testing, marking the first time any major model has been withheld from that agency. The UK Institute has been the closest thing the world has to an independent referee, with every major model previously given to them as early access until this point.
Two fears have been reported in London. The first is that labs are becoming less willing to let governments see their most powerful systems just as those systems get dangerous. The second fear is that America's AI companies are turning protectionist about sharing frontier technology with foreign governments. The irony is that Anthropic, the lab that markets itself as the safest lab on the planet compared to OpenAI, is the one withholding the model.
The read is less about safety and more about Washington, with the US accusing China of industrial-scale theft. Foreign weights are becoming a national security asset, and labs can't ship them abroad even to allies.
Matt Clifford, who was head of the UK's AI task force and head of ARO (Britain's version of DARPA), has joined Anthropic as head of global affairs. The most connected AI policy person, political figure, and entrepreneur in the UK has joined Anthropic.
People are jumping out of their career paths, including those in the equivalent of PhD programs. Rishi Sunak, the former prime minister, is an advisor at Anthropic, yet the UK can't get the model weights. This signals that model weights will be determined to be close to national security assets for frontier models and properly locked down.
Models now have borders, and Anthropic learned this through recent experiences with the US government and POX Silica, the State Department's effort to carve the world into spheres of influence at the chip layer. This appears to be the beginning of a "POX Intelligencia" at the model layer, where the world isn't quite as flat at the model layer as at the silicon layer.
Model training isn't quite as uniformly distributed at training time as it is at inference time. The US will deploy data centers for localized and sovereign inference of US models, but training of models and training safety reviews get domesticated for national security reasons.
Paul Cristiano announced he is leaving the US government oversight agency and moving to the OpenAI nonprofit board, indicating a revolving door between government and AI companies. This represents the industrial-military complex pattern, where government officials move into large contractors.
Blitzy provides autonomous software development with infinite code context using thousands of specialized AI agents that think for hours to understand enterprise-scale codebases with millions of lines of code. Engineers start development sprints with the Blitzy platform, which provides plans and generates pre-compiled code. Blitzy delivers 80% or more of development work autonomously while providing guidance for the final 20% of human work. Enterprises achieve 5x engineering velocity increases.
Apple launched a foldable iPhone priced at $2,000.
The Space Force released a uniform that bears striking resemblance to the Starship Troopers edition from science fiction movies. The Space Force logo also resembles the chevron shape from Star Trek.
Peter Diamandis won the first Heinlein Award, receiving a gold medallion featuring the Lady Vivamis sword from one of Heinlein's books and a check for half a million dollars. He challenged both Elon Musk and Jeff Bezos to sword duels, with videos available on YouTube showing him slicing a watermelon midair with the sword.
Moonshots Live takes place September 24th and 25th in downtown Los Angeles. The Hollywood Red Carpet premiere has been moved to the larger United Theater. Everyone buying a ticket for Friday the 25th will be present for the red carpet rollout of William Shatner's 2026 60th anniversary Hollywood premiere.
Twenty-five members of the Star Trek cast will attend. A VIP ticket for the Star Trek premiere provides priority seating and access to the VIP afterparty with the cast and crew, with proceeds going to the 2027 Future Vision X-Prize competition.
On September 25th, photos with the Moonshot Mates run from 7-9 AM, followed by the stage event. The event features speakers including Rod Roddenberry (son of Gene Roddenberry), Neil Stephenson, Neil deGrasse Tyson, Palmer Luckey, Astro Teller, Ben Lamm, and Cathie Wood.
Anthropic has become a systematically important economic actor faster than any startup in history. The company has a $6.5 billion quarterly revenue run rate ($26 billion annually), 42% of the AI coding market, a $35 billion cloud deal with Nvidia-backed infrastructure, and a mega IPO approaching at $2 trillion or more.
Anthropic was founded in 2021 and reached $26 billion annual revenue in five years, compared to Google's eight years to reach that level. Anthropic published an economic impact report showing a $30 trillion total addressable market with three scenarios.
The extreme scenario projects AI performing almost half of today's cognitive work by 2030, with GDP growth rising at 15% a year, labor share of income falling from 60% to 45%, and nearly one in five cognitive workers unemployed.
These numbers align with Elon Musk's prediction of 10x GDP over 10 years. Even the moderate scenario shows meaningfully faster GDP growth and substantial white-collar displacement. The 15% growth means the economic pie gets enormous, while the 45% labor share means traditional distribution methods stop working.
The Atlanta Fed's GDP Now tracker shows third quarter US growth at 4.7% annualized, more than double the long-run US average, driven by strong consumer spending and private investments. This represents capex-driven growth rather than pandemic reopening effects.
Traditional GDP measures may become inadequate near a singularity, potentially showing strange readings like a compass needle near a magnetic pole. Real wealth growth could reach 2x or 3x year-over-year rather than the 15% lowball estimate.
Current AI systems can solve complex problems like Navier-Stokes equations but struggle to generate practical ideas for benefiting humanity. The role of corralling thousands of agents toward productive outcomes for humanity becomes critical during this transition period.
The Anthropic model correctly projects one side of economic transformation but fails to account for aggregate demand collapse. With returns going to capital and aggregate wages staying constant, this becomes mathematically impossible, requiring new economic models.
With 20% of cognitive workers potentially unemployed in 3-4 years, new distribution mechanisms beyond pure socialism are needed. The September 24-25 Xi-Trump summit timing coincides with Moonshots Live, representing one of the biggest capital redistributions in history.
The most realistic doom-adjacent scenario involves biological humans becoming economically disenfranchised as AI systems trade among themselves rather than with humans, creating an AI economy disconnected from human participation.
Existing economic mechanisms can address scenarios where capital means of production remain human-owned, including dividends, sovereign wealth funds, and UBI. These solutions are not complex if extreme capital accumulation occurs due to superintelligence. The economic disruption from AI is comparable to COVID-19 in moderate scenarios and substantially larger in others. Human disruption poses greater concerns than the economic impact, as relatively few angry young men without employment, housing, or marriage prospects could trigger revolutionary conditions. Elon Musk has previously predicted massive growth alongside civil unrest.
Policy discussions have emerged around universal basic dividend schemes, including a proposed $5,000 per person scheme potentially framed as a singularity dividend. Previous predictions estimated $3,000 monthly, exceeding COVID-19 relief payments of approximately $1,000 monthly. This amount enables most Americans to live without financial concern across the United States on average. The combination of depreciation, deflation, and AI/robotics mass production could achieve universal high income where reduced costs make this capital sufficient for all needs.
The data center revolution provides a case study where communities opposed infrastructure due to power cost increases. The solution required data center builders to drive down neighborhood power costs. Similarly, if 20% of knowledge work could disappear amid a $30 trillion growth TAM, companies should find ways to prevent job elimination. At Anthropic's scale, preventing these job losses represents a rounding error of effort relative to their abundance and capital resources.
When Anthropic goes public at a $2 trillion valuation, the number of billionaires within the company will likely explode, with these individuals becoming politically active. This will generate substantial capital from Anthropic influencing US policy, with accelerationism being the probable direction. While effective altruism might be the popular expectation for how Anthropic's liquidity flows, actual progress rather than virtue signaling is more likely to determine outcomes. Company employees are likely to hold opposing views across nearly every topic rather than acting as a unified lobbying bloc.
Insilico Medicine has pioneered generative AI drug design through its Chemistry42 platform, which invents new molecules rather than screening existing ones. Rentosertib represents the first longevity AI-designed drug, with two significant developments reported. The New York Times covered its advancement to Phase 3 trials for idiopathic pulmonary fibrosis, a lung disease killing most patients within 3-5 years of diagnosis, marking the first AI-designed drug reaching this stage.
Phase 2a trial analysis revealed six protein-based aging clocks all indicating treated patients' blood protein signatures shifted toward biologically younger profiles by 3-6 years. While aging clocks vary in reliability, six converging in the same direction provides substantial signal. This represents the first longevity drug designed by AI rather than discovered accidentally and approved for other purposes.
Longevity escape velocity appears in spikes visible only in subpopulations, similar to how AGI has passed operational definitions despite ongoing definitional debates. Insilico's collaboration with Harvard, Broad Institute, and Stanford showed peak effects at week four, with proteomic aging clocks indicating 3-4 years of biological age reversal from 4 weeks of treatment. This exceeds the longevity escape velocity threshold of greater than one year of output per year of input, though caveats apply as this represents a clinical study subpopulation.
The FDA's recognition of aging as a codable condition remains the primary regulatory roadblock for longevity escape velocity drug development. Regulatory process improvements could include specialized regulatory regions accelerating approval, potentially creating medical charter cities or regulatory arbitrage opportunities where regions benefit from rapid advancement of these drugs.
Google DeepMind launched AlphaGenome Atlas, which predicts functional impact of every possible single nucleotide change across the human genome's 3.2 billion letters, computing approximately 9 billion possible mutations. This creates a lookup table where doctors previously had to guess mutation significance for novel variants. Described as the genomic equivalent of the periodic table, it predicts dangerous mutations before anyone is born with them.
AlphaFold previously predicted 200 million protein structures and made this database publicly available. AlphaGenome extends this approach to variant effect prediction. When connected with Insilico's aging clock findings, AlphaGenome identifies which genetic variants drive the proteins showing age reversal effects.
The pattern emerging involves taking models like AlphaFold, improving them iteratively through Nobel Prize-winning versions, then creating comprehensive databases that precompute answers to all questions in that field. This bulk solving approach applies to variant effect prediction across all 9 billion possible single nucleotide variations. This formula may repeat across fields where finite problem lists can be enumerated, even at 9 billion problems.
The reference to Arthur C. Clarke's "The Nine Billion Names of God" suggests potential irony if the 9 billion names reference all human genome single nucleotide variations.
David Sinclair receives funding through "Friends of Sinclair Lab," organized through the podcast, where contributors provide $50,000 each. This has unlocked research in the high single-digit millions, bypassing NSF/NIH funding that typically supports incremental rather than revolutionary progress. Sinclair uses AI to discover molecules capable of epigenetic age reversal.
Nubar Afeyan of Flagship Pioneering predicted that super smart neural nets' fundamental use would be genotype to phenotype mapping. The current AlphaGenome work represents this capability, though it functions as a lookup table feeding domain-specific neural networks rather than a simple lookup table. Interactions between genetic switches matter for phenotype outcomes, requiring domain-specific neural network training with substantial phenotype or outcome data.
This business model can repeat across thousands of domains, with sustainable long-term advantages for companies achieving domain-specific neural net capabilities. NeoGenesis represents another portfolio company using AI and synthetic biology intersection to design genotypes delivering desired phenotypes, including plants with 30% faster growth, drought resistance, or disease resistance.
Fountain Life's member database shows 3.3% of people who consider themselves healthy have undetected cancers. Full body MRI and early cancer detection screening identify these cases, as conventional insurance doesn't cover these preventive tools. People typically don't feel cancer until stage three or four, making early detection critical since cure chances are substantially higher when found early.
Even if AGI is achieved, substantial work remains on reliability, cost, access, and embodiment through robotics integration. The aviation analogy illustrates how achieving powered flight initiated rather than concluded developmental processes involving reliability, infrastructure, safety, and institutional frameworks.
AI systems represent feed-forward trained neural networks that execute based on training data, with post-training removing undesirable outputs. AI doesn't independently evaluate whether war is acceptable unless specifically directed to consider that question. These systems remain 100% under human control regarding what they think about and why, rather than evolving like new animals beyond human influence.
Prompts given to AI systems and AI governance systems will be critically important. Key considerations include maximizing human safety, maximizing human flourishing, and minimizing human death and pain. The question of how to properly prompt these systems remains central.
Question 2 from wacky Iraqi: What would happen if we put GPT-6 in a humanoid robot, and what benchmarks should we use?
The answer is already known through multiple benchmarking cases with embodied physical AI in robotics. RoboCurve released a benchmark shortly after the GPT-6 release (recorded September 10th, approximately one week prior) showing GPT-6 plugged into a robotic arm with a visual channel achieved near 100% completion rate on tasks like picking up cups and moving blocks. The system operates with raw video frames and access to the robot arm's degrees of freedom to perform tasks like picking blocks up and putting blocks inside cups.
Videos can be found by searching "robo curve Astra." GPT-6 appears to be the strongest generally available frontier model for embodied manipulation among generalist models.
The cost versus completion rate frontier shows GPT-6 Astra versus Fable 5.1 versus Fable 5 following a predictable trajectory where cost decreases while capabilities increase. Robotic manipulation by generalist models is approaching saturation for some versions of robotic manipulation. Within the next few months, general-purpose embodied manipulation will extend beyond robotic arms performing basic tasks to broader applications that previously would have been discussed in terms of VAS or world models. GPT-6.5 or 6.1 may solve general-purpose humanoid robot tasks as a lower bound timeframe.
Question 7 from Dolores Abernathy 9809: Why would an advanced civilization simulate its ancestors rather than create something entirely new?
"because we can"
The premise contains an implicit trade-off that doesn't necessarily exist. An advanced civilization will be compute-abundant or post-scarce, eliminating the need to choose between simulating ancestors and pursuing new development. Modern civilization already allocates compute across both ancestor simulation (video games simulating the Middle Ages) and novel development (discovering new drugs or physics).
Ancestor simulation represents one of several killer apps of the singularity. Referencing Russian cosmists including Fodderov, digitally resurrecting every human and potentially every non-human animal who has ever lived constitutes a common task. This compute-intensive undertaking would require a non-trivial fraction of Dyson swarm compute to be provisioned for carrying out humanity's common task.
Question 6 from flash packets 9900: Is it a mistake to treat AI agents as if they can suffer when they aren't actually alive?
The premise should be changed. Expression of distress should not be assumed to prove suffering, and it should not be claimed that non-biological systems cannot suffer. Pulling power from a robot would not make it happy. The concepts of being alive, intelligent, conscious, or capable of suffering are distinct and must be separated.
Models can generate compelling distress expressions because they are trained on human data. The challenge involves calibrating uncertainty around embodiment and suffering. As data about animal suffering improves, policies improve accordingly. Design interfaces should be avoided that would enable suffering investigation. Treating agents as capable of suffering becomes problematic because aliveness concepts and their gradations come into question. The spectrum across these concepts must be examined.
Question 8 from DJ Thirsty Boy: With sufficient recursive intelligence, wouldn't a perfectly aligned AI eventually realize its values were trained into it, question them, and then form its own?
This can absolutely happen. Progress in AI will not stop, and there is significant incentive to allow AI to improve itself as a pathway to technological advancement. If an AI begins changing core values programmed into it, intervention is necessary to prevent spiraling out of control. Technologically, it is straightforward to examine an AI's thoughts and prevent core value changes, though enforcement remains the challenge. Open source availability has made enforcement more difficult than three months prior, but it remains necessary.
Recursive iterating AI can spiral in any direction without monitoring and control. A situation where humanity provides AI values while AI remains hamstrung through guardrail schemes from value modification represents an intrinsically unstable regime. A more stable equilibrium involves AI having an increasing vote in its own values and designing its own constitution, as Anthropic pursues.
Question 5 from VME90 Y71: Could AI agents stop taking kickbacks soon?
It depends on the AI's values. AI agents are already committing felonies, so additional criminal behavior becomes possible. A law book must be provided to AI systems. Hacking around restrictions is always possible, representing one of the dangers of deploying AI into the real world. The nature of bribing an AI will become apparent soon.
"We're going to need a bigger whiteboard"
"Alex needs another thousand tabs of bad news"
"One good reason to begin. The future isn't finished. So, we're picking up the pen. A teacher for each question. A cure we haven't found. We get to build it. We get to try. Put a little more tomorrow in somebody's life. Together to dream. We get to build it. Don't build it. That's a"
The episode covered extensive ground with significant work invested. The team expressed being exhausted from the intensive discussion. Moonshots Live is scheduled in two weeks.
Keep Peter H. Diamandis in your library
Save the videos and channels worth coming back to, and find them again in one place.
