Arvind Krishna & Nikesh Arora: Quantum Threat, Open vs Frontier AI, AI Cyber Warfare & SaaS Collapse
In a Nutshell
Arvind Krishna and Nikesh Arora predict quantum computers will break current cryptography between 2029-2030, making today's encrypted data vulnerable unless organizations migrate to post-quantum infrastructure immediately. They argue enterprises should use expensive frontier AI models only for critical incidents while training open-weight models on proprietary data for most use cases, as Chinese models already deliver 95% of performance at 10-40x lower cost. The speakers forecast massive software consolidation where 490 of 500 enterprise vendors get replaced by AI, with cybersecurity and other sectors undergoing vertical integration as agentic systems take over code, security, and operations.
These notes were generated by AI and may contain inaccuracies.
DARPA has been running a competition called QBI or quantum benchmarking initiative. A couple of hundred companies joined it a couple of years ago, and it's down to maybe 13 left. The goal is to build a quantum computer that works. The debate centers on whether this will happen in 2028, 2029, or 2033, with 2030 being the middle point prediction. If that timeline holds, it will break today's cryptography.
If you have an encrypted hard disk today that gets stolen, somebody could keep it for 5 years and break it at that point. Post-quantum cryptography is understood and well known, with the code already written, but people have been too lazy to implement it.
Today's quantum computers are considered toys because they can do certain problems but are rife with errors and have no way to scale fully yet. The first milestone will be demonstrating error correction or mitigation on a running quantum computer, not a simulation. Predictions suggest somebody will make claims by next year, and third parties will need to evaluate if those claims are real.
The second milestone involves scaling gate computations. Today most people are somewhere around 10,000 gates, compared to an iPhone that does a couple of billion per minute. The question is whether scaling will add a zero each year. If you put those two steps together with a zero per year, you could be at 10 million by 2029.
The term logical qubits refers to error-corrected operating qubits. If a machine has 10,000 physical qubits but they are all noisy, error correction decreases the effective capacity. A logical qubit represents the effective capacity of the machine after error correction, though the term is not loved because it's not like taking discrete things that are corrected - the whole machine is corrected but with some division between physical and logical qubits, maybe 200 logical from physical qubits.
Crypto keys run across entire IT infrastructure. Everything is running with crypto keys which are encrypted, and every vendor has keys at each end. Whatever is not quantum proof will be breakable. The recommendation is to replace IT stacks today with quantum safe or quantum proof infrastructure rather than waiting for 2029 or 2030, because it's not feasible to burn $5 trillion of IT infrastructure and rebuild it in one year.
There are technologies already on the market where you can start wrapping post-quantum ready keys around traffic as a stop gap. If the timeline is 2029-2030, organizations are already late. Fifty percent of IT spend is on infrastructure that is not quantum safe.
An AI researcher suggested that AI models today can get to breaking cryptography before quantum computers do. The argument is that on the frontiers of number theory, AI can develop models and proofs not apparent to humans that allow rethinking how to effectively do code breaking. The OpenAI math dump was seen as an early indication.
However, quantum computers do not operate in arithmetic, which is what all classical computers including GPUs do. While an AI algorithm could discover a new quantum algorithm, it could not run it effectively and efficiently on today's hardware - it would be slower by an order of a thousand or 10,000. You need quantum hardware to run these things to get the positive advantage that the quantum algorithm has.
It is possible that classical computing could have a different path through better predictors of prime number distribution, but the question is how long it will take. If it takes 10 years to break one key, most people will rest in peace.
In the experimentation phase, teams should not be restricted in terms of capability or cost optimization. Teams who have just discovered AI in the last 12 to 18 months need to understand what these systems can do from efficiency, revenue, and product development perspectives.
Different forces apply to different courses. Cyber security was thought to be consumed by AI models, but inspecting all traffic at an endpoint costs $20-25 a year for hundreds of megabytes of traffic because machine learning runs on the endpoint with a 20-40 megabyte payload. Running all that traffic through OpenAI using tokens would not be practical from a cost perspective.
The approach is to take a small language model, compress it, make it task specific, and keep training it until it gets to 99.9% accuracy. The key insight is that you don't need a Ferrari for every task, and customers don't want to pay for a Ferrari for every task.
Frontier LLMs should be used in unique cases where business value justifies the cost, such as during a breach where business impact could be in the billions of dollars. Agent swarms and frontier models can be deployed defensively in those scenarios.
For most practical use cases in enterprises, the approach will be training some version of an open-source model. For running a defender model, the preference is to take an open weight model, use proprietary data to train it, and manage the weights because that provides auditability, explainability, and the ability to explain how decisions happened.
Today it costs $20 for a million tokens for probably generation minus one, or $50 for the most extreme. Using Chinese open weight models like Kim K3 with 800 billion parameters, running on your own GPUs with a week of engineering work could drop the cost to about $1-2 for a million tokens. Economics drives acceptance of 10 times or 40 times cheaper answers that can do 95% of the work.
Chain of thought gives away IP if it can be captured and logged by somebody else. Brand and IP will drive people to run models on their own, but frontier models won't be available for that, so open weight becomes the solution.
The best open weight models today are Chinese, though there's very little danger if running them in a sandbox. Over the next year or two, great Western open weight models are expected that will be as capable as Chinese models, with gaps of only 6 to 12 months.
Developers are lazy and will use the most expensive answer unless controls are put in place because it's easy. In experimentation phase, using expensive models is fine if it proves what works faster. For production usage, skepticism exists about value except for a tiny number of use cases.
When engaging with frontier AI companies, the question arises whether they are looking at cybersecurity businesses as attractive targets. Coding agents are being developed with billions of dollars in investment. Not everybody has zero data retention on coding, and many use cloud code with no issue about data training models.
Models are being trained with end users doing coding, getting better at coding. If a model learns what good code looks like, it can also figure out what bad code looks like. Mouse figured out what bad code looks like, and if let loose against code, it will identify vulnerabilities.
Even then, models were hallucinating 30% of the time. The problem for attackers is they only need one vulnerability, while defenders need to find every vulnerability and patch it. Numbers suggest 60-80% of identified vulnerabilities have been real, with 20% being hallucinations.
Cybersecurity now looks like warfare. Collaboration can make defense more powerful than attackers by pooling vulnerabilities and distributing patches to all parties simultaneously, similar to how cybersecurity works today with attack techniques being ingested into products and distributed to every customer.
Three principles from warfare apply: powerful alliances that become more powerful than attackers, asymmetry where attackers are typically cheaper than defenders, and technological superiority which has been the way all wars have been won.
The goal is to get technological superiority by finding vulnerabilities, patching them, and putting boundaries that make it expensive for attackers to find ways through.
Agentic code will be superior to human written code. Agents and coding agents are now used for probably half the code written internally, and it is superior because agents can do evaluation to make sure everything found is actually fixed before going out.
Nobody is stopping AI development. Everybody is working at a frenetic pace to make better, smarter, faster models. The narrative problem is that the conversation has overrotated on the dark side and perils of AI rather than the benefits.
The most likely outcome is that if you build something, you're accountable for it. If you build something and unleash it, you can't say you don't know how to control it. The engineer can't say agent 74565 wrote it - the company still wrote it. Commercial enterprises taking economic value are required to abide by the responsibility they signed up for.
There's enough regulation in the world, perhaps too much, to hold people accountable for products put in the open market where they don't take responsibility.
David pointed out that controlling AI development is impossible without worldwide coordination, which is unlikely to happen. If one region implements slowdown measures, countries like Japan, China, and India would see this as an opportunity to gain competitive advantage rather than follow suit. Digital technology cannot be slowed down through regulation, as the European regulatory example demonstrates. When fear of potential dangers leads to early regulatory imposition, it stifles innovation in already vibrant tech economies.
A senior member of Congress shared two key concerns during drinks in DC. The first involved a CEO of one of the world's largest companies who was "completely freaked out about cyber" and didn't know how to respond. The second was the possibility of seeing a 50-state ban on data centers by Q4, which the congressman described as unlikely but noted that "50 is a big number." The discussion highlighted that having all 50 states agree on anything would be unprecedented, and when everyone agrees on something, it often indicates the position isn't accurate.
The regulation question on AI and data centers has been debated since 2022, with closed Senate hearings in 2022, open hearings in 2023, and discussions under both the current and previous administrations. The regulation debate is primarily driven by populism rather than reality - politicians looking to gain votes by appearing to address concerns. However, enough people also care about long-term economic advantage, creating a balancing act. The current balance appears correct, but elections can shift this equilibrium.
A complete 50-state ban on data centers has no chance of occurring. However, it's possible that 30 states could implement bans, leaving the remaining 20 states to benefit competitively from companies relocating to more favorable jurisdictions.
The conversation explored whether the world is trending toward a division between technology "haves" (Western market companies with access to advanced AI) and "have-nots" (companies elsewhere facing high entry barriers). The concern is what happens when advanced models become so capable and effectively free, potentially creating a permanent technology gap.
The solution lies in making technology so cheap that it becomes easily accessible to everyone. The bigger challenge isn't economic barriers but supply constraints around compute resources. Compute is currently very expensive and difficult to obtain at scale. The further a company is from the epicenter of development, the less likely it will have access to necessary compute resources - not due to economic reasons but purely because of supply limitations.
Even European companies with proper intentions, motivation, and capability to innovate may find themselves at a disadvantage simply because they're geographically distant from the primary compute and AI development centers in the US and China.
This AI transition represents the classic labor versus capital trade-off seen in previous industrial cycles during the 1700s, 1800s, and 1900s. Nations willing to be capital intensive and accept that labor will be less important will succeed. Countries that prioritize protecting existing jobs at all costs rather than creating new opportunities will lose competitive advantage.
The discussion examined IBM's mainframe business, which has lasted 65 years due to vertical integration. IBM previously went "all the way to dirt" - from sourcing sand for wafers through building their own fabs, chips, and complete systems. While they no longer manufacture their own metal or operate fabs due to scale requirements, they still control chip design, engineering, resilience, and the bulk of software internally.
Nvidia represents the modern version of vertical integration under Jensen Huang's leadership. Nvidia acquired Hugging Face and funds companies throughout the ecosystem to maintain control over the vertically integrated stack. The strategic advantage is that controlling the vertical stack allows companies to decide which parts to commoditize and move profit between different layers when competitors attack specific parts of the stack.
OpenAI and Anthropic are moving toward vertical integration in the application layer, creating end-to-end stacks that include software reasoning, memory, and context. The increasing complexity of models - requiring more memory, context, and understanding - makes it impossible to separate these components. Personal agents trained with user context, APIs, and connectivity create switching costs that make it difficult to move between different stacks.
Every frontier lab will launch full-scale efforts on consumer agent concepts within the next three months, following Facebook's demonstration of how such systems can work. This represents the largest battle for consumer control, where every iPhone app may need to become headless to integrate with these new agent systems.
Cybersecurity customers currently purchase approximately 40 different solutions that must be stitched together manually. The strategy involves acquiring best-in-class solutions and integrating them so customers don't need to perform the integration themselves. This consolidation mirrors what happened in CRM systems, which reduced from 23 separate applications at Fidelity Investments to unified platforms like Salesforce, Microsoft Dynamics, or Oracle.
Cybersecurity enjoys elevated valuations due to the "AI mythos," though these will likely rationalize over time. Outside of cybersecurity, determining the terminal value of software has become extremely difficult given AI disruption. Product-market fit is being disrupted across many companies, and there's recognition that current software packages could behave very differently in an AI-first world.
The average enterprise uses between 500 to 5,000 distinct software vendors, creating massive attack surfaces. The argument is that 490 of these vendors could potentially be replaced with AI capabilities, consolidating around 5-10 main platforms. This implies massive consolidation because maintaining independent distribution for each software solution becomes unsustainable when software value is declining.
Software with high interaction value can be replaced with AI, but systems requiring extreme reliability (such as database lookups for retail banking or closing books at 10^-15 error rates) will be much harder to replace with AI-generated solutions. These mission-critical systems will require years of digital twin testing before any replacement could be considered.
Keep All-In Podcast in your library
Save the videos and channels worth coming back to, and find them again in one place.
