Why Hardware-Software Co-Design Is AI's Real 100x: Dylan Patel of SemiAnalysis
In a Nutshell
Dylan Patel argues that hardware-software co-design across model architecture, systems software, and silicon delivers the real 100x efficiency gains in AI, far outpacing isolated improvements at any single layer. SemiAnalysis built InferenceX as a living benchmark platform to track these cross-layer optimizations, revealing that models like DeepSeek are explicitly shaped for specific chips (Hopper, Blackwell, TPU) while the CUDA moat has shifted downstream to model optimization. The core bottleneck remains memory bandwidth and power density stuck at ~1W/mm², with future scaling requiring co-packaged memory, space-based compute by 2040, and differentiated infrastructure for batch versus latency-sensitive workloads.
These notes were generated by AI and may contain inaccuracies.
SemiAnalysis has 90 people with a large portion being technologists and engineers across the semiconductor supply chain, alongside former hedge fund professionals. The team engages in organic debates where engineers push for the coolest technology while others emphasize cost considerations. The environment is informal and Dylan Patel enjoys moderating these discussions.
Sean from Sequoia introduces Dylan Patel at the SemiAnalysis office alongside partner Sonia Huang. Five years prior, semiconductors were not considered sexy in the West though they remained attractive in the East. Dylan went very long on the sector and built the premier research company in the space, covering technical details, supply chain, and the bigger picture. Rumors indicate SemiAnalysis recently passed $100 million in revenue.
Dylan grew up in a small business environment where his parents owned a motel and gas station. He jokes that his first neural network training involved racially and visually profiling customers entering the gas station to determine which cigarettes to recommend. The cigarettes were displayed across the top of the counter, requiring him to move a step stool based on customer demographics including age, profession, and race.
On his 8th birthday in May, Dylan asked for an Xbox 360 for Christmas instead of a birthday gift. When his cousin from Alabama visited during spring break, Dylan wanted to impress him but the Xbox experienced the red ring of death hardware defect. Dylan opened the device and shorted the temperature sensor to fix it after trying other methods unsuccessfully. This experience opened Pandora's box for him regarding hardware.
Sign in to read the full notes
Get access to AI-generated notes, topic timestamps, and more.