Dylan Patel — The Single Biggest Bottleneck to Scaling AI Compute
In a Nutshell
Big Tech's $600B CapEx this year deploys ~20-50 GW of AI compute, but the single biggest long-term bottleneck to scaling beyond 200 GW/year by 2030 is ASML's EUV tools (limited to ~100/year), constraining TSMC's advanced node production amid exploding HBM/DRAM demand that triples prices and halves smartphone volumes. AI labs like OpenAI/Anthropic race to lock 5+ GW via aggressive multi-provider deals (Nvidia, CoreWeave, Oracle) at $2-2.40/GPU-hour, prioritizing unipolar fleets for massive models while power/labor/data centers prove solvable via modularization, turbines, and behind-the-meter gas. US leads China in compute/revenue scaling (10 GW/lab next year), but Huawei could dominate sans bans; space GPUs remain impractical due to deployment delays and chip limits.
These notes were generated by AI and may contain inaccuracies.
Combined forecasted CapEx of big four—Amazon, Meta, Google, Microsoft—is $600 billion this year. Yearly prices of renting that compute equate to close to 50 gigawatts. Not all coming online this year; paying for compute coming online over coming years.
OpenAI raised $110 billion, Anthropic raised $30 billion. Compute coming online this year for labs on order of four gigawatts total. Cost to rent compute that OpenAI and Anthropic will have this year is $10 to $13 billion per gigawatt. Raises cover yearly compute spend, excluding revenue.
$600 billion CapEx across supply chain totals order of $1 trillion. Portion for compute online this year (chips, parts). Much is setup CapEx. 20 gigawatts incremental capacity added this year in America; portion not spent this year, some from prior year.
Google's $180 billion: chunk on turbine deposits for '28/'29, data center construction for '27, power purchase agreements, down payments for future scaling. Applies to all hyperscalers.
Anthropic and OpenAI at 2 to 2.5 gigawatts now, scaling larger. Hyperscalers' biggest customers are Anthropic and OpenAI.
Anthropic: $4-6 billion revenue added last few months. Straight line: add $6 billion/month, implying $60 billion revenue next 10 months. At reported gross margins, $40 billion compute spend for inference. At $10 billion/gigawatt, need 4 gigawatts inference capacity (assuming flat R&D training fleet). Need >5 gigawatts by year-end; tough but possible. OpenAI similar or higher.
If Anthropic not on track for 5 gigawatts for revenue + R&D, where from? Dario (Anthropic) conservative on compute to avoid bankruptcy if revenue inflects differently. But revenue crazier than expected; OpenAI more aggressive, signed crazy deals, has more access.
Anthropic must go to lower-quality providers (vs. historical Google, Amazon). Microsoft expanding. OpenAI aggressive: Microsoft, Google, Amazon, plus CoreWeave, Oracle, SoftBank Energy (new to data centers), NScale.
Conservatism led to financial freakouts last year (fears OpenAI couldn't pay). Now raises prove they can.
To acquire excess compute: hyperscalers have capacity. Not all long-term 5-year deals; shorter 1-3 year, on-demand from 2023/2024/2025 H100s. As contracts roll off, highest bidder wins. H100 prices up; long-term deals >$2, up to $2.40 for 2-3 years. Hopper build cost $1.40 over 5 years; high margins.
Neoclouds (e.g., CoreWeave, Together AI, Nebius) had more Hopper, shorter deals. Some unsold capacity or reallocated from internal use.
Anthropic can use hyperscaler compute via Bedrock (Amazon), Vertex (Google), Foundry (Microsoft) with revenue share (50% markup equivalent) or last-minute spot compute at premium.
OpenAI faced "no deals" for months due to funding doubts, now open. Few incremental buyers yet; best models (Anthropic) enable deal-locking at better prices.
Bears (e.g., Michael Burry) argue 2-3 year depreciation due to rapid tech improvement, inflating amortized CapEx.
TCO model: H100 $1.40/hour over 5 years (data center, networking, spares, chips, servers). $2/hour deal = ~35% gross margin.
Counter: new chips (Blackwell, Rubin) drop H100 spot price ($2 → $1 → $0.70). But supply limits mean pricing based on utility today, not comparison.
GPT-5.4 cheaper/smaller/sparser than GPT-4, better quality, more tokens/GPU on H100. GPT-4 TAM few-$tens billion; GPT-5.4 >$100 billion (with lags).
H100 worth more today than 3 years ago. With AGI/human-level, H100 (~1e15 FLOPs, brain-equivalent) could produce 6-figure value/year, repay in months.
Dario says 2-5 years to data center of geniuses (trillions revenue), inconsistent with conservative compute vs. OpenAI.
Value/GPU rises over time; commit now. Only OpenAI/Anthropic see it now, but soon all will.
Alchian-Allen effect: fixed cost increase (e.g., GPU price $2→$3) reduces relative premium for better model, pushes to best (all volume/revenue on best models).
Committers lock 5-year compute at old prices, margin advantage. Short-term flex capacity at modern high prices.
Most compute in long-term (>3 years for CoreWeave 98%). New capacity transacts at new prices.
Incremental compute costs drive; infra providers (Nvidia, memory, clouds) capture margins. Nvidia: $90 billion long-term contracts, 3-year memory deals. Amazon/Google via Broadcom/AMD. Memory prices doubling/tripling.
Model vendors margins up this year due to capacity constraints, demand destruction.
Nvidia ~70% N3 wafer capacity by '27. Fractures neoclouds/RL/data to avoid lock-in.
TSMC gives N3 (Trainium 3, TPU v7) to Nvidia (vs. fracturing). Last year 3nm mostly Apple (moving to 2nm). Memory prices up → Apple volumes down or margins cut. TSMC higher margins on HPC/AI vs. mobile. Apple to third-parties as squeezed. TSMC providing really good
TSMC prioritizes allocations to companies doing CPUs over AI chips. Amazon's Trainium (AI chip) and Graviton (CPU) are both on 3 nm; TSMC is more excited to allocate to Graviton due to stable, long-term growth in CPUs. Conservative foundries allocate to stable, lower-growth markets first before incremental capacity to fast-growth markets like AI. Same for AMD: TSMC prefers their CPU allocations over GPUs. Nvidia is unique with CPUs, switches, networking, NVLink, InfiniBand, Ethernet, NICs—most on 3 nm by end of year with Rubin launch (GPU most important). Nvidia gets majority of supply due to market forecasts and signals: non-cancelable, non-returnable commitments, deposits signed earlier than Google or Amazon. Google/Amazon had delays (e.g., one chip delayed couple quarters, Trainium issues). Nvidia demanded more aggressively, checked supply chain (e.g., Victory Giant, Chinese PCB supplier; all/many PCBs from China; memory vendors).
AGI-pilled companies buy compute on long timelines at high levels, paying margins now despite future distortions. Nvidia not fully AGI-pilled (Jensen calls it accelerated computing, not just AI; broader than AI, includes physics modeling/simulations; not embracing automation like Dario or Sam). Still more AGI-pilled than Google/Amazon in Q3 last year, saw more demand via data center construction tracking. Google/Amazon (especially Google) deploy GPUs despite better TPUs due to fab shortages.
Google sold ~1 million v7s (Ironwoods) to Anthropic. Bottleneck is logic/memory for chips. Why sell instead of using for DeepMind? Google Cloud/execs prioritized differently; Anthropic compute team (both main people from Google) negotiated deal exploiting dislocation before Google realized. Early Q3: TPU capacity surged multiple times in 6 weeks; Google explained sudden increase to TSMC (much for Anthropic). Anthropic anticipated; then Nano Banana and Gemini 3 skyrocketed Google's metrics. Leadership woke up, stated doubling compute every 6 months. Went to TSMC for more but sold out—maybe 5-10% more for 2026, focus on 2027. Information asymmetry: Anthropic/Fluidstack signed ahead; Google screwed up (evident from Gemini ARR: near-zero Q1-Q3, slight Q3 inflection, $5B Q4 ARR). Google more conservative than Anthropic (less ARR initially). Now Google absurdly AGI-pilled: bought energy company, turbine deposits, massive powered land, utility long-term agreements.
Bottlenecks shift yearly: couple years ago CoWoS, last year power. Long-term (5 years out to singularity): semiconductor supply chains have longest lead times (fabs 2-3 years vs. data centers <1 year, e.g., Amazon 8 months). Fabs more complex. Tools have long lead times. Early bottlenecks (CoWoS, power, data centers) shorter lead; capacity slid from mobile/PC to data center (now exhausted). Nvidia now largest customer at TSMC and SK Hynix. No more sliding from consumer; scaling AI chip production is biggest bottleneck to 2030.
By 2028-2029, bottleneck is ASML (EUV tools): most complicated machine, $300-400M each. Current: ~70/year; next year 80; aggressive expansion to little over 100 by decade end. Translates to AI compute via gigawatts. Rubin (announced GTC, end-of-year release): 1 GW needs 55,000 3 nm wafers, 6,000 5 nm wafers, 170,000 DRAM wafers. Key: EUV lithography (30% advanced logic cost). 3 nm wafer: ~70 masks/layers, 20 advanced EUV. 55,000 wafers × 20 EUV passes = 1.1M passes; total ~2M EUV passes/GW. EUV tool: ~75 wafers/hour, 90% uptime → ~3.5 EUV tools/GW. 1 GW costs ~$50B; 3.5 tools = $1.2B. TSMC $100B CapEx over 3 years ($30/30/40B); Nvidia fraction for 3/4 nm. Nvidia Q? $40B revenue → $160B annualized. Nvidia turns small CapEx fraction (depreciated over years) into $160B/year; intensifies downchain to ASML ($1B machines/GW, last >1 year).
~250-300 EUV tools now + 70 this year, 80 next, to 100/year → ~700 by decade end. 700 ÷ 3.5 = 200 GW AI chips (if all to AI, not). Sam Altman wants 1 GW/week (52 GW/year) = 25% share (reasonable; this year ~25% Blackwell GPUs). ASML shipping EUV ~decade, mass volume ~2020 (initial lower throughput). Tools improve: overlay (layer alignment after deposit/etch/clean steps), throughput. Price $150M initially → $400M by 2028, but capabilities >doubled (throughput, overlay). ASML generous: no crazy price/margins despite monopoly (China maybe EUV by decade end). Matches capability increases, net benefit to customers.
ASML not YOLO expanding; semi supply chain avoids booms/busts, not AGI-pilled (don't see trillions spend). Numbers deemed too high, then validated but next year still "too high". Tool components: source (Cymer, San Diego), reticle stage (Wilmington, CT), wafer stage, optics (Europe). Complex supply chains not massively expanded; long lags. Source: tin droplets hit 3x laser (expand, shape, blast → 13.5 nm EUV light); collected/directed to lens stack (Carl Zeiss primary, multilayer molybdenum/ruthenium mirrors; ~18/tool; artisanal, defects ruin). Reticle stage: 9G acceleration (scans 26x33 mm sections, steps; wafer complementary, opposite directions). Marvel of chemistry/fabrication/mech/optical engineering; extreme metrology. Built in Eindhoven, deconstructed/shipped/reassembled (months). >10,000 suppliers. Accuracy: sub-1 nm (for 3 nm overlay). No snap scaling; specialized people (e.g., Zeiss <1,000). Supply chain lag: Nvidia/Anthropic/OpenAI short despite pushing; chain does X-1 or X/2. Whip reaction slow; think 60→100 enough + tool improvements (500W→1kW source).
Elon wants 100 gigawatts a year in space by 2028 or 2029. Sam Altman wants 52 gigawatts a year by the end of the decade. Anthropic and Google probably need the same. The supply chain can't build enough capacity for everyone to get what they want on the compute side.
People argue the data center supply chain is bottlenecked by specific things, limiting AI compute scaling. If the grid is a bottleneck, alternatives include behind-the-meter on-site power, gas turbines, etc. Similarly, in semiconductors, if EUV becomes a bottleneck, could revert to 7 nm using multi-patterning with DUV machines, like China does.
A100 (7 nm) is 312 teraflops FP16. B100 is a little over 1 petaflop FP16, a 3x improvement holding numerics constant. Some from process improvement (7 nm to 4 nm), some from accelerator design. 150k wafers per month at 3 nm, similar for 2 nm, and similar for 7 nm. With 50% haircut due to fewer bits per wafer area, adding 7 nm wafers could provide 50-100 gigawatts. Possible if desperate for incremental compute despite higher cost/power.
Comparisons unfair: A100 optimized for FP16/BF16. Hopper for FP8. Rubin for FP4/FP6. New 7 nm chip optimized for modern numerics still lags significantly beyond raw FLOPS difference. FLOPS per watt/dollar oversimplifies.
On optimized software, Hopper vs Blackwell: vast performance differences for eight-bit models. Blackwell's FP4 not fully utilized here, yet gulf much larger than FLOPS/numerics suggest.
Models run on hundreds of chips (e.g., DeepSeek production on 160 GPUs). Crossing chips incurs efficiency loss via SerDes (latency/power cost). Shrinking nodes increases on-chip data movement to tens/hundreds TB/s, vs ~1 TB/s inter-chip, ~100 GB/s inter-rack (400-800 Gbit/s). Hierarchy: on-chip super fast, intra-rack 10x slower, inter-rack 10x slower. Hopper vs Blackwell: 20x inference perf difference (100 tokens/s for DeepSeek/Kimi K2.5), not 2-3x FLOPS, due to networking/etc. Some architectural improvements portable to 7 nm, but not all; compounds with FLOPS/chip, networking, memory bandwidth.
B200: two dies/chip for intra-chip bandwidth without NVLink/InfiniBand. Rubin Ultra: four dies. Limits on die count for tens TB/s. Advanced packaging (CoWoS by Nvidia, Google, Broadcom, MediaTek, Amazon Trainium). Tesla Dojo (cancelled/restarted): wafer-sized with 25 chips, no HBM, best for CNNs but poor for transformers. Larger packages constrain networking, memory bandwidth, cooling. Trend: more chips/package possible on 7 nm (Huawei Ascend 910C/D: 1→2 dies). But doable on leading-edge (3 nm) too.
By 2030, West has advanced process but possibly less ramp-up; China mass produces (maybe EUV/2 nm). Crossover when West process edge fades, China scale/indigenization wins. China not fully indigenized yet; 7 nm/14 nm use ASML DUV. ASML revenue (esp. EUV) outside China. China scale advantage still West/Taiwan/Japan/Korea. China trying indigenous DUV/EUV.
Bullish: by 2030, fully indigenized DUV (yes), EUV working tools but not mass production (production hell like ASML's early 2010s→5-7 years ramp). China maybe ~100 DUV tools/year vs ASML's hundreds. No node at 1M wafers/month (Elon/China/TSMC aspire; memory makers maybe not single fab).
SemiAnalysis to model when indigenized Chinese production > West combined, inputting DUV/EUV timelines. On 2035 timelines (not long-scheme), expect China dominating semis? Fast takeoff: US wins; slow: China wins via vertical supply chain vs West's fragmented (US/Japan/Korea/Taiwan/SE Asia/Europe).
China models competitive recently, but Opus 4.6/GPT 5.4 pull away. Shift to hidden-reasoning automated work harder to distill from US models. US labs: OpenAI 2 GW end last year → 10 GW end next; Anthropic 2+ GW this year → 10 GW next. China not scaling AI compute as fast. US CapEx: Amazon $200B, Google $180B, $1T total US data centers this year. High ROIC: Anthropic $4B Jan revenue, $6B Feb (compute-constrained, low Claude reliability). $20B ARR, sub-50% margins → $13-14B rental compute ($50B CapEx). 10x revenue likely; China lacks deployment scale. Fast compounding revenue drives US growth; China lags infrastructure.
If middling returns (e.g., Google $300B CapEx next year wrong), US overbuilds; China scales vertical chain, overtakes on longer timelines.
HBM has 3-4x fewer bits/wafer than base DRAM. Could future accelerators use commodity DRAM for more capacity? For async agents, tolerate lower latency/bandwidth (HBM stacks for bandwidth).
Possible: 4-5x cheaper Opus 4.6, 2x slower (existing HBM curve). But no demand; high-value tasks time-sensitive (hours → day unacceptable). DDR challenges: chip I/O edges (sides HBM 2048 bits, top/bottom inter-chip); DDR lowers bandwidth, raises capacity—but care about bandwidth/wafer, not bits/wafer (weights/KV cache constrain FLOPS). Four constraints: FLOPS, network BW, memory BW, capacity. DDR shifts design, wastes FLOPS waiting on memory; can't increase batch (KV slower).
HBM4 (Rubin): 2048 bits over 13 mm wide, 10 GT/s → ~2.5 TB/s/stack. DDR same area: 64-128 bits, 6.4-8 GT/s → 64-128 GB/s. ~10x less BW/edge. Chip max 26x33 mm die limits edge area; bandwidth constrains despite caching.
4x more wafer area same bytes → destroy 4x consumer demand (laptops/phones) for 1 AI byte. 2026: 30% Big Tech CapEx (~$600B total) to memory.
Nvidia engages in margin stacking, so separate their margin from memory and logic. A third of their CapEx goes to memory.
Memory crunch will continue to get harder, prices will continue to go up. Affects different market parts differently. People will hate AI more because smartphones and PCs won't get incrementally better year on year; they'll get incrementally worse.
iPhone bill of materials (BOM): 12 GB memory. Each GB used to cost $3-4, so $50. Now price tripled to $12 per GB for DDR, so $150 vs $50—$100 increase for Apple. Apple has margin, won't eat it all. NAND has same dynamics, so likely $150 increase on iPhone BOM, leading to $250 more for consumer. Lag due to Apple's 3-month to 1-year long-term memory contracts. Apple hit hard, adjusts at next iPhone release. High-end market: Apple sells 200-300 million phones annually.
Bulk market is mid-range/low-end. Annual smartphone sales: 1.4 billion previously, now 1.1 billion, projections 800 million this year, 500-600 million next year. Data from China analysts in Asia, Singapore, Hong Kong, Taiwan: Xiaomi and Oppo cutting low-end/mid-range volumes by half. For $1000 iPhone, $150 BOM increase absorbable due to margins. Smaller phones: higher % of BOM to memory/storage, lower margins, no long-term agreements.
Smartphone volumes halving hits low/mid-range hardest, not high-end. Consumer devices >50% of memory demand. Even halving smartphones, low-end cut >half, high-end <half (consumers still buy >$1000 phones despite price hikes; Apple volumes drop less). Same for PCs. DRAM released to AI chips willing to do long-term contracts, pay higher margins (larger end-user margins). Leads to more AI hate: memes on PC subreddits/gaming Twitter blame AI for doubled memory prices blocking gaming GPUs/desktops. Will worsen with DRAM prices doubling again.
NAND prices also rising; capacity expanded slowly (almost zero last few years). Higher % NAND to phones/PCs than DRAM. Destroying demand (mostly DRAM) unlocks more NAND for other markets. DRAM price increases > NAND because more released from consumer + more produced for AI.
SSDs used in data centers, but not as large quantities as DRAM/HBM.
Same constraints as logic scaling: EUV tools needed for memory. Currently not EUV bottleneck (that later decade); fabs not built past 3-4 years due to low memory prices/losses in 2023. Market recovered slowly, boomed 2024. Predicted reasoning needs long context → large KV cache → high memory demand 1.5-2 years ago; AI experts went long on memory. Took 1 year for pricing reflection, 3-6 months more for fab starts (fabs take 2 years). Meaningful new fabs late 2027-2028.
Micron bought fab from Taiwan lagging-edge chip company. Hynix/Samsung expanding existing fabs with crazy measures, large economic knock-ons. No place to put tools—not just EUV, other tools bottlenecked (logic N3: 28% wafer cost EUV; DRAM: teens %, rising). Other tools (Applied Materials, Lam Research) expanding, but fabs (most complex buildings) take 2 years.
Elon plans TeraFab: 1 million wafers/month (no fab that big). Clean rooms bottleneck this/next year.
Elon could build facility faster with 'move fast' approach? Clean rooms complicated but possible faster. Doesn't solve tooling bottleneck in 2 years.
Constraints shift; even non-bottlenecks have margins (e.g., energy grows fast post-bottleneck). Clean rooms biggest bottleneck 2028-2030. Elon excels at garnering resources/smart people via crazy projects (Mars, reusable rockets, autonomous EVs, humanoid robots)—unique recruiter. Semis: possible for 1M wafers/month. Step 1: clean room doable in 1-2 years (not super fast initially, accelerates). 'Dirty rooms' mindset wrong: air replaced every 3 seconds, ultra-low particles.
Hard part: developing process tech/wafer building (built-up knowledge). Most complex: TSMC/Intel/Samsung integration.
2030 total EUV disruption (simpler, higher volume alt)? Very low probability. Particle accelerators/synchrotrons for 13.5nm/X-ray 7nm light complicated/massive. Some companies could disrupt beyond EUV, but no magic simple direct-write high-volume.
Hope: 3D DRAM like 3D NAND, revert to DUV? Roadmaps still use EUV for tighter overlay in vertical stacks. 3D drastically ups bits per EUV pass. Roadmap: 6F cell → 4F cell → 3D DRAM end-decade/early next. Likely, but needs R&D/manufacturing/integration. Requires huge fab retooling: different tool breakdown (CVD, ALD, dry etch, chemistries); lithography least changed. Can't quickly convert logic/DRAM/NAND fabs. Even 1α→1β→1γ needs DUV/chemistry changes/EUV.
Litho % wafer cost up: 2014 17% → 30% (DRAM low-mid teens → high teens, soon 20%). 3D DRAM tanks EUV % wafer cost.
Jensen/Sam Altman can't just demand from TSMC; long-run bottleneck ASML/tool/material makers (even with Intel/Samsung expansion). SV should pitch ASML now for 2030 tools?
2023-2025: Energy foresight bought turbines from Siemens/Mitsubishi/GE Vernova, now premium pricing. Similar for EUV, but ASML won't trust randoms (turbines cheaper/more produced). Put $1B deposit for 10 EUV tools in 2 years, resell at premium once shortage hits. ASML gets demand signal, but supply chain limits ramp. Arbitrage like TSMC: lock capacity via forward contracts, sell later at insane margins ASML/TSMC should've charged.
Power can scale (not arbitrarily). Blog: AI labs → 60 GW/year gas turbines (GE Vernova, Mitsubishi, Siemens). 2030: 200 GW/year logic/memory enough? From 20-30 GW critical IT now (gross up 20-30% for losses/transmission/cooling; capacity factors; PJM models 20% excess, turbines 90% derated). Nameplate >> actual IT.
Beyond turbines: aeroderivatives (airplane engines → turbines; Boom Supersonic/Crusoe); medium-speed reciprocating (10 makers; Cummins; auto mfg surplus → data centers; diesel→gas); ship engines (Nebius for Microsoft NJ data center); Bloom Energy fuel cells (fast production ramp, higher cost); solar+battery; wind+battery (15% factor).
Grid scaled for 10-20% peak spikes (hot summer days). Utility-scale batteries/peaker plants (gas/industrial turbines/combined-cycle/batteries) unlock 20% US grid (idle most time). Data centers 3-4% US power now → 10% 2028. US grid terawatt-level. Hard engineering/risks/new tech, but simpler supply chains than chips. Elon pioneered behind-meter gas; explosion since.
Specific turbine blade lead times >2030? Many other ways; inefficiency fine. Combined-cycle CapEx $1500/kW; alternatives up to $3500/kW (2x) → GPU TCO +few cents/hour (Hopper $1.40 → $1.50 if power doubles). Models improve fast, marginal utility >> cost.
20% grid (200 GW) from batteries etc. Gas vendors: 16+ manufacturers, hundreds GW orders tracked. End-decade: ~half capacity behind-meter (more expensive; grid issues: permits/queues).
Even though it's more expensive, people are doing behind-the-meter power. Options include reciprocating engines, ship engines, aeroderivatives, combined-cycle (not great for behind-the-meter), Bloom Energy fuel cells, or solar plus battery. Any of these individually could do tens of gigawatts, and as a whole, hundreds of gigawatts.
Electrician wages will probably double or triple. In Abilene, at the 1.2-gigawatt data center that Crusoe is building for OpenAI, 5,000 people worked there at peak. Scaling to 100 gigawatts would require 400,000 people (assuming efficiency gains). There are 800,000 electricians in the US and millions of construction workers. Adding 200 gigawatts a year could crunch labor. Labor is a humongous constraint; people need training. High-skilled labor from Europe (e.g., decommissioning power plants) will be imported.
Humanoid robots or robotics will help, but main reductions come from modularizing in factories in Asia (Korea, Southeast Asia, China). Ship built-out sections: e.g., 2-megawatt blocks integrating high-voltage AC to DC power delivery to racks, or fully integrated cooling subsystems (plumbers are a constraint). Ship entire skids with rows of servers, networking, cooling, power integrated. Today, single rack is 120-140 kW; next-gen like Nvidia Kyber approaches 1 MW. Entire row reduces cabling, networking fiber, power connections, plumbing. This drastically reduces on-site labor.
Crusoe, Google, Meta are advancing modularization. Faster adopters may face delays; slower ones face labor shortages. Complex supply chain causes dislocations, but solvable through capitalism and human ingenuity on required timescales.
Elon Musk bullish on space GPUs. Power not a constraint on Earth; permitting for hundreds of gigawatts cited as issue.
America is big; data centers don't take much space. Air pollution permits challenging, but Trump administration eased them. Texas skips red tape. Elon faced issues in Memphis (idled appliance factory, aluminum refinery connection), built power plant across border for Colossus 1 and 2. Texas allows more flexibility; site chosen for grid power, water, gas access, electricians. With 50 states, small jurisdictions allow temporary workers (3-12 months) in housing, paid highly—labor cheap relative to GPUs, networking, token value. Diversifying to Australia, Malaysia, Indonesia, India; still 70%+ AI data centers in America.
Dealing with permitting in Texas, Wyoming, New Mexico easier than space. Energy small fraction of total cost of ownership (TCO). Power free in space, but even if Earth power doubles, still fraction of GPU cost.
ClusterMAX rates neoclouds (tested 40+ clouds, including hyperscalers). Differentiation in deployment and failure management. GPUs unreliable: 15% Blackwells RMA'd. Elon argues low failure after initial phase, but testing/deconstructing/shipping to space/launch/redeploying takes months—10% of 5-year GPU life. Compute most valuable first 6 months for models/revenue. Some Earth clouds take 6 months to deploy; others faster. Space would take longer than Earth testing.
Starlink satellites: 100 Gbps inter-satellite, potentially higher with optical lasers (close to InfiniBand 400 GB/s per GPU, but racks have 72 GPUs; scales with Blackwell/Rubin). Inference often splits across scale-up domains; sparse models ping experts across hundreds/thousands chips. Needs satellite interconnects (expensive, 15-20% cluster cost). Space lasers vs. high-volume pluggable transceivers (unreliable, need frequent cleaning/replugging).
Space not limited by energy (can build 1 TW/year globally). Chips bottleneck at 200 GW/year end-of-decade. Deploy chips ASAP on Earth via modularization/racks (pre-wired except chip).
By 2035, semis (ASML, Zeiss, Lam Research, Applied Materials) catch up; optimize energy (10-15%, up to 30% with ASICs, lower Nvidia margins). Elon wins with 10x gains, not 20%; space 10x eventually when resources contentious, but not this decade.
1 W/mm² now for AI chips; to 2 W/mm² = 20% more tokens/wafer (exotic cooling: cold plates, liquid, immersion—solved on Earth, hard in space). Hotter chips radiate per Stefan-Boltzmann law (4th power), but density requires exotic cooling harder in space.
Scale-up: tight domain with TB/s communication. Within chip fastest; rack TB/s; far hundreds GB/s; cross-country GB/s.
H100 server: 8 GPUs at TB/s. Blackwell NVL72: rack-scale, 72 GPUs all-to-all at TB/s (doubled speed, 8→72 scale).
Thousands chips: TPU v4 (4k), v7/v8 (8-9k). Torus topology: 6 neighbors, bounce for distant comms (blocking).
In-between: larger domains, all-to-all via switches + torus. All moving to dragonfly topology (fully connected elements + partial, scales to hundreds/thousands without full blocking).
Claim: slow parameter scaling (GPT-4 >1T params; now approaching) due to Nvidia scale-up memory. 5T model FP8 + KV cache ≈ 10 TB for forward pass. GB200 NVL72: 20 TB first Nvidia scale-up; prior smaller. Google TPUs had hundreds TB. Partially explains via capacity/bandwidth; larger models slower to deploy.
Allocate to inference (revenue), development (pretrain/RL), research. Research efficiency gains huge: models 10x cheaper/year at same scale. Prioritize research to push scaling laws. Development: larger models need more RL rollouts (5T = 5x vs 1T; even if 2x sample efficient, 2.5x RL time). Smaller model + more RL faster; deploy sooner to research/dev. Feedback favors smaller models.
Google deploys largest (Gemini Pro > GPT-5.4, Opus) via unipolar TPU (leverage 1k chips scale-up for fast RL). Anthropic: H100/H200/Blackwell/Trainium/TPUs. OpenAI: mostly Nvidia → AMD/Trainium. Unipolar optimizes large models; multi favors smaller/faster cycles. Compounding: smaller → faster research → efficiency wins → faster takeoff.
SemiAnalysis sells spreadsheets (warned memory crunch 6-12 months ago; now cleanroom crunch, future tool crunch). Leopold uses for outrageous money; others say numbers too high (hyperscalers deny competitors' scale). 60% business industry (AI labs, data centers, hyperscalers, semis); 40% hedge funds. Leopold unique in saying numbers too low. Data makes markets efficient; exploit via constrained surprises. Leopold has most AGI takeoff conviction.
Bets exemplified by public returns for funds like Leopold's are about what happened in the last year, predictable using spreadsheets, reports, API access to data—buying the next year's spreadsheets. It's not about singularity; it's about buying the memory crunch, which requires believing AI will take off hugely. Memory crunch predicated on KV cache exploding as context lengths get longer, needing more memory, plus supply chain understanding of fabs, data centers, chips. A year ago, predicting memory prices quadrupling while smartphone volumes drop 40% over next year or two sounded crazy, but believers traded memory successfully—not just Leopold, who sized and positioned well, but others too.
N3 mostly AI accelerators, N2 mostly Apple now, but AI will want N2. TSMC unlikely to kick out Apple if Nvidia, Amazon, Google offer high prices due to long chip design timelines (>1 year out). Instead, Nvidia et al. prepay for capacity, TSMC expands slightly, tells Apple they project need for X minus one when ordering X. Apple's flex capacity gets screwed. Apple traditionally over-orders 10% and cuts back 10%, varying by season/macro. Apple will become smaller percentage of TSMC revenue, less relevant; TSMC may require pre-booking 1-2 years out with prepay for CapEx, as Nvidia/Amazon/Google do.
N2 Allocation Details
This year, Apple has majority of N2 (includes A16 variant), little from AMD's early AI/CPU chips. Next year, Apple ~half as others ramp, then falls drastically like N3 where they were half. Traditionally first to nodes, but 2nm first time not (besides Huawei previously). AMD risking delays with brand-new 2nm for CPU/GPU chiplet + advanced packaging to scale faster than Nvidia. On A16 node, first customer is AI, not Apple. Apple no longer first or majority; becomes any old customer as TSMC CapEx balloons while Apple's business grows slower. Apple cuts orders due to rising costs in packaging, materials, DRAM, NAND—can't fully pass to consumers.
If Huawei had 3nm access, potentially better accelerator than Rubin. Huawei first with 7nm AI chip (Ascend), two months before TPU, four months before Nvidia A100; also first 5nm mobile chip. Process node leadership doesn't imply full software/hardware, but Huawei has all legs: software engineers, networking (historical biggest business), AI talent/researchers better than Nvidia's in concentration via China pool, own fabs, own end-market for tokens. Argue Huawei with TSMC would beat Nvidia. China advantages in scale, optical tech Nvidia can't access easily. If not banned 2019, Huawei would eclipse Apple as TSMC's biggest customer via share in networking, compute, CPUs.
If humanoids take off fast (millions by 2030), each needing local compute: don't need all intelligence on robot—inefficient vs. cloud batching. Cloud handles planning/long-horizon tasks at high batch sizes, pushes directions; robots interpolate actions or handle short tasks like "pick up cup" (weight/force on-device). Robot sends sensory data (e.g., "headphone"), cloud identifies ("Sony XM6s"), provides details (soft headband, weight); robot acts on inputs. Cloud instructions every second or 10x/second. On-device full processing expensive (no batching), less intelligent (smaller models), diverts leading-edge chips from data centers amid shortage—power bad for robots, needs low-power efficiency. Millions of humanoids reduce 200 GW data center power.
Intelligence centralizes physically: from 8B humans' heads to data centers driving robots/knowledge work.
Elon recognizes, signs massive Samsung deal for robot chips in Texas for Taiwan risk, separate supply chain. Samsung has little AI demand besides Nvidia's new LPU (launching next week). Provides geopolitical/supply chain diversity, less competition vs. TSMC's infinite data center demand.
If tools ultimate bottleneck, airlifting all TSMC process engineers (if blockaded) insufficient: fabs not all in Taiwan now (risk); tools use Taiwan-made semis (snake eating tail, though not super advanced). Destroying fabs strengthens China's verticalized chain. Replicating know-how/capacity in Arizona etc. takes long, slashes US/global GDP massively, incremental compute near zero—from hundreds GW/year end-decade to 10-20 GW via Intel/Samsung. Existing capacity pales vs. expansion, causing crazy AI dynamics.
Keep Dwarkesh Patel in your library
Save the videos and channels worth coming back to, and find them again in one place.
