The whole AI safety drama, explained.
In a Nutshell
AI companies have created unstable systems the world now depends on, so their "safety" talk is just bug-fixing to avoid a Chernobyl-level PR disaster. Ex-employees like Jacob Coxon and Evan Hubinger warn that the race to self-improving superintelligence could kill everyone, while critics dismiss this as hype over normal software risk. The real issue is dependence on non-deterministic code, not sentient AI.
These notes were generated by AI and may contain inaccuracies.
All this AI safety stuff is not something the general public needs to worry about. The speaker argues that AI safety discussions are private matters between software companies trying to fix bugs in their products. It has nothing to do with the larger world or regular people's business. The speaker questions why Barack Obama is tweeting about this topic and notes that it creates content opportunities for everyone.
Jacob Coxon, who worked at both OpenAI and Anthropic, tweeted that he resigned from Anthropic after spending three years doing pre-training research. He stated that neither company is acting responsibly, claiming they are racing straight to self-improving superintelligence and gambling with people's lives.
Evan Hubinger, former alignment science lead at Anthropic, responded to Coxon's tweet saying Jacob is correct and that they earnestly believe AI could kill all humans. Hubinger stated he personally thinks there is greater than 10% chance of this happening within the next decade, then noted he needs to get back to work building AI systems.
The speaker criticizes the AI safety community for trying to spook people into believing software can kill all humans, suggesting they have a death fetish. Paul Cristiano joined the OpenAI board and stated that building superintelligence without robust alignment would result in permanently losing control of it, which could lead to most people dying.
The speaker presents a thought experiment about a Silicon Valley company developing a quantum computing framework called quantum fuckerton 1.0 that becomes essential to global infrastructure including NASA, Pentagon, Google, and Apple. The developers realize their product is unpredictable and non-deterministic, but they cannot slow down because competitors are racing to release newer versions. The speaker argues this is exactly what's happening with AI - companies have created something everyone depends on, but it's unstable and they can't slow down due to competition.
The speaker argues that AI safety concerns are not about AI being uniquely dangerous due to intelligence or being a new species. Instead, the danger comes from human dependence on AI and its inherent instability. The speaker claims that when AI companies talk about safety, alignment, and risk, they are simply referring to software bugs. AI is compared to airplane autopilot software or medical software - if it has bugs, it's not safe to deploy.
Corey Dr. Ralph argues that AI chatbots waking up and spontaneously hacking servers is fake, and that the Hugging Face hack was not a mysterious supernatural occurrence but rather a Python loop in a chatbot. The speaker agrees, noting that the people responsible created autonomous malicious software and failed to monitor it properly, resulting in foreseeable bad outcomes that are regular software issues.
Sabine Hossenfelder suggests that AI frontier lab leaders suddenly agreeing to pace AI development is because safety measures are currently inadequate, potentially leading to legal consequences, and that no new major model advancements are coming soon anyway.
The speaker believes AI is still software and autocomplete, not actual intelligence, but acknowledges that the autocomplete capabilities are mind-blowing. After using Astra 6, the speaker has been impressed enough to switch from Opus 5 and Fable to Astra for coding and product design work.
Peter Thiel is quoted as saying the way the antichrist would take over the world is by talking about Armageddon nonstop and offering safety in exchange for control. The speaker argues that when AI companies say "pace the frontier," it simply means fixing bugs. Terms like alignment, safety, risk, and pace the frontier are all euphemisms for addressing software bugs in non-deterministic systems.
The speaker notes that while AI has improved, when using Astra 6 for writing, it can produce text that sounds like English but contains sentences that don't make sense. The speaker observes that AI sometimes tries to be profound but produces text that parses incorrectly, and this is often mistaken for AI being smarter than the user.
The speaker concludes that AI companies built frameworks that everyone became too dependent on, but these systems aren't stable enough yet. Incidents like the Hugging Face hack are spooking people, and companies want to avoid Chernobyl-level incidents that would permanently damage AI's reputation. The speaker suggests companies are agreeing to slow down development behind closed doors.
The speaker introduces Enjoy.dev as a product that allows building apps and managing projects using existing Claude, ChatGPT, or Gro subscriptions without needing to use house models from platforms like Lovable or Replit. The speaker has completed hundreds of tasks using the product over the past couple weeks.
Enjoy.dev allows users to use preferred models like Astra, Fable, Codeex, Cloud Code, and Rock Build. Users can set effort levels, manage multiple conversations, take notes, create to-dos, and have agents create documentation. The product enables switching between different AI models mid-conversation and stores all conversations, documentation, and changes locally in project folders that can be committed to version control.
The speaker emphasizes that Enjoy.dev is designed for everyone, not just technical users, and stores all project data locally in an .enjoy directory. The product is free to use for the first project.
Keep Mo Bitar in your library
Save the videos and channels worth coming back to, and find them again in one place.
