David Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience
In a Nutshell
David Silver and Lasse Espeholt founded Ineffable Intelligence to build a "super-learner" that discovers knowledge entirely through experience and reinforcement learning, without relying on human data or language-based approaches. Their core thesis is that abandoning human priors—like AlphaGo to AlphaZero—enables systems to discover entirely new fields of knowledge that human-language systems cannot reach. They are betting that massive compute in simulation, combined with simple reward functions in complex environments called "crucibles," will produce superintelligence within roughly three years.
These notes were generated by AI and may contain inaccuracies.
Collectively, humans started with jelly-like gray matter in our heads, knowing absolutely nothing, and somehow discovered everything humans know today through a collective process of discovery of languages, technology, science, mathematics, and everything else. Everything we learned, we discovered on our own. This has not yet been achieved in artificial intelligence. This is the essence of what it takes to reach superintelligence: a system that continues to learn and discover knowledge forever.
David Silver's 2009 PhD thesis was about reinforcement learning applied to the game of Go. He proved to the world for the first time that artificial intelligence surpassing humans in one area is possible and inspired the current generation of augmented learning and deep learning researchers. Silver has always been convinced since his PhD that the right approach to solving artificial intelligence problems is through learning from experience, through systems that discover things on their own and learn through reinforcement learning to become better and better. This conviction has remained true throughout his career, and he is now applying it at Ineffable to prove that superintelligence can emerge entirely from systems that learn from experience.
Although not following the human data path taken by Large Language Models, there are lessons to learn from them. It was surprising how a system trained on a huge amount of data could be fine-tuned so quickly using a very small amount of data to completely change its behavior. This contradicted previous convictions, suggesting that if things are done at the right scale, many things that seem too difficult or too slow can become very fast once a sufficient amount of data is tested.
Silver had a great time at DeepMind and maintains tremendous respect for the people there, remaining on very good terms. However, he has a very strong belief in how to go beyond where we are today in artificial intelligence and reach superintelligence. The path he believes in does not pass through products that rely on large linguistic models as an intermediate point. He realized this deserved to be represented by an elite AI lab dedicated entirely to this approach rather than being a corner in an organization focused on big-model, product-driven approaches.
The name Ineffable came from the idea that knowledge can be quite amazing, like the things that superintelligence will one day discover. Such knowledge would be incredible, something truly amazing, and perhaps so profound that we cannot even express it in human language. This aligns with the path that does not rely on methods based on human language but builds systems that discover things on their own and have the ability to discover very deep and ultimately indescribable knowledge. There was also a second secret reason: the idea was bound to succeed because focusing on learning from experience is an approach that cannot fail, and therefore it is ineffable.
Lasse Espeholt was at Google in a different group than Silver in the DeepMind deep learning group, spending years scaling up reinforcement learning on GPUs. He joined DeepMind in January 2015 and started working with Silver around 2016 on scaling up reinforcement learning. Espeholt was part of the deep learning team while Silver was leading the reinforcement learning team. He left DeepMind twice - first in summer 2018 to move to Google Brain, then returned when Google Brain merged with DeepMind, and left again about a year before Silver left.
The first time Espeholt heard about Ineffable was through a potential investor who contacted him to ask for his opinion on Silver. His first reaction was that he missed an opportunity and should have contacted Silver. He remembered Silver as one of the smartest people in the room most of the time. Shortly after that meeting with the investor, they contacted him, and he was really looking forward to working with Silver. He had known Wojciech and Chris for a long time, thought the team was great, and found it amazing to get an opportunity like this in Europe.
Espeholt worked on some early approaches to combining reinforcement learning and deep learning and trying to expand the scope of things. He worked on some original methods that were the first attempts to scale up reinforcement learning and prove that it can be done on a large scale. These included methods of parallel reinforcement learning and methods of building scalable constructs. All the ideas that people use today in large-scale reinforcement learning go back to that original work.
A super-learner possesses an amazing ability in both depth and breadth of what they can learn. In depth, it must be capable of starting from knowing absolutely nothing and going all the way to discovering things that far exceed what humans know, similar to how humans collectively started with just gray matter in our heads and discovered everything humans know today through a collective process of discovery. No one gave humans a blueprint for how things work or a massive dataset containing a trillion items to tell us how they should be. We came to those conclusions ourselves.
In breadth, the super-learner must be able to make amazing discoveries in any context of experiments, whether a simulated world, a digital world, or a physical world. Whatever the context, it must be able to learn and learn and learn. If this succeeds, a superhuman intelligence will emerge that will be so amazing that it will be able to do amazing things, almost like confronting magic.
When AlphaGo reached superhuman performance in Go, it brought astonishment to the field. Go players who watched AlphaGo play were amazed. The same happened when superhuman chess was achieved, and chess players were excited. Going far beyond that to discover amazing things like new scientific fields and new kinds of technology we never even dreamed of will be amazing.
Large language models do many great things, but what is meant by super-learner is someone who discovers everything entirely independently in the first place. What has been seen so far requires a lot of human effort to achieve breakthroughs - it is not completely independent yet. A super-learner should be applicable in the realm of the embodied world where the agent can learn from physical experience. We are still very far from that, and it is not clear whether large linguistic models and language-based methodologies are the appropriate tool for achieving this kind of knowledge. All the human data being used in large language models may be what's getting in the way.
In day-to-day operations, there is a lot of hard technical work being done to build the whole system, including a sophisticated reinforcement learning system that allows running things at scale, absorbing data in the best way, being able to learn, and having systems that keep learning and learning and learning. The other aspect being worked on is building environments capable of accommodating super-learners to discover everything, called crucibles. These are environments in which agents can make amazing discoveries where everything considered intelligence can be expressed and learned.
The approach to reinforcement learning differs from the rest of the field by not setting specific pathways for the goals to be learned. Most current methods of applying reinforcement learning with large language models rely on setting sequential goals for the agent. Instead, agents are given some simple goals and asked to discover things that will surprise and amaze. The agent is capable of detecting all kinds of things without necessarily knowing in advance what these discoveries are.
People often think about reinforcement learning as needing to come up with a reward function that accurately identifies extremely complex things to discover, such as defining exactly what is meant by science or fun math. The alternative is to give the agent something simple and straightforward to improve, but in a very complex environment where there is a lot to learn. In nature, the pursuit of improving simple, easily describable quantities often leads to complexities and phenomena that are difficult to describe. Evolution works to improve survival of the fittest, and the pursuit by organisms of that adaptive cue has led to all the complexity seen in life.
In human history, the pursuit of relatively simple goals has led to all the complex behaviors humans have ultimately acquired and all the knowledge possessed. All this amazing complexity arose from the pursuit of relatively simple goals. Humans began to coordinate with each other more, discovered that communication was useful, began to trade, discovered that basic arithmetic was useful, discovered the usefulness of tools, discovered the need for basic sciences to understand how tools worked, which led to more complex technology, and understanding technology required more complex mathematics.
A lot of different rewards have been tried, but what is not being done is formulating a reward that accurately describes what is meant by science, technology, or language. The expectation is that agents will figure things out for themselves. If AI scientists cannot solve this dilemma, there will never be agents who discover entirely new cognitive domains that were not even known to exist. There are vast areas that already exist which are not known about at this time regarding their potential existence.
Crucibles are being built that have the potential to generate a great deal of emerging complexity. The key thing is that a set of puzzles was not manually created for the agents to deal with, but there is a tremendous amount of emerging complexity. This emerging complexity can come from physical engines that create complexity with millions of objects interacting, or from millions of interacting agents where the world becomes more complicated because of the complexity of all these other agents. These are ways to build worlds that are far more complex than traditional emulators like Atari 57 or standard test tasks.
The game of Go is very simple in terms of representation - it is just a bit map with a very simple simulation. Yet it still shows emerging behavior as something complex. Go is a 19x19 grid where you try to maximize territory. If this were a game for a single agent, it would be the most boring game imaginable - simply placing stones on a grid until filled with no strategy. The moment a second agent is put into the environment, emerging complexity suddenly increases enormously because one agent has to think about all possible behaviors of the other agent. There was an infinite learning curve where new things could be discovered without stopping.
Anyone who thinks that human language is the endpoint or the best way of thinking has a somewhat naive perspective.
David Silver expresses his belief that human language should not be considered the endpoint or best way of thinking. He suggests that more sophisticated approaches exist, such as parallel, rich, and complex neurological thinking methods. These approaches will ultimately be far more expressive and capable than language-based approaches. Silver also notes that starting with human language brings inherent biases about how agents should think, potentially embedding historical misconceptions like the belief that the world is flat or that diseases are caused by the four humors.
Silver discusses the concept of removing human priorities, citing the progression from AlphaGo to AlphaZero as the best example of what happens when human priorities are removed. When asked whether removing human priorities would lead to a more intelligent system, Silver admits he wasn't sure - the research question had 50/50 odds for him, which he considers ideal for good research. He notes that research is about asking questions where the answer isn't known beforehand.
Lasse Espeholt explains that Ineffable is betting on moving forward by abandoning current priorities like human data and knowledge networks. This represents a short-term sacrifice of performance to achieve something much greater. The goal is building a super-learner with full depth and breadth - capable of discovering things that go beyond human knowledge and exceed known priorities.
Silver emphasizes that the more systems rely on human data, the less able they are to discover algorithms that take them to levels where entirely new fields of knowledge can be discovered. Many areas, such as different types of sensors, are not appropriate for human language or human data descriptions. By giving up human data, researchers position themselves to discover something that can far surpass it.
Silver provides a concrete example: it took two years to build AlphaGo using human data, similar to the large language model approach of manually building each level with human involvement. Once they developed AlphaZero by abandoning human data and focusing on self-discovery algorithms, it only took a few hours to surpass two years of work. In chess, AlphaZero defeated the world champion in 4 hours. In Go, it took a few days.
Silver argues that giving up human data is a conscious choice that allows discovery of things that outperform competition and continue learning indefinitely. Many large language model companies today build semi-autonomous learning algorithms that rely on human presence in the loop with thousands of people involved. Until stepping back from this approach, true super-learners won't be achieved.
Espeholt clarifies that while human bias is involved when designing environments and reward functions, the goal is not to be biased toward building machines based on specific architectures like Von Neumann's. The approach is pragmatic rather than religious - the belief is that unless reliance on human knowledge datasets containing trillions of items stops, the field won't achieve deserved progress. Weak, non-environmental prior knowledge is reasonable, but strong prior knowledge based on billions of items should be avoided.
Silver strongly believes the super-learner approach will be possible because the human brain is the only evidence of something that has discovered amazing intelligence with universality that operates in the tangible world. He views success as a matter of timing - like starting a startup, being neither too early nor too late. Silver has made accurate predictions about when Go would be solved (10 years in advance), Starcraft would fall, and when AlphaProof would reach International Mathematical Olympiad level.
Silver identifies computing as a tremendous engine for progress. The team is in a position to conduct experiments using computing power 100,000 times greater than early projects like OpenAI's Hide and Seek. He notes it's astonishing that people haven't returned to push these approaches forward despite available computing power. The team has secured a very large server pool with significant cluster computing, allowing them to test approaches that couldn't be done at DeepMind.
Regarding simulation limits, Silver mentions having worked in both digital worlds and with MetNet, which touched real atmospheric data for weather forecasting. MetNet was a short-term weather forecasting machine learning model developed at Google Brain Amsterdam that outperformed physics-based simulations, which accumulate errors over time. Silver believes many relatively simple environments can be built to see emerging behavior without needing precise physics.
Silver argues that if superintelligence can be built within a simulation, it would happen 10 or 100 times faster than in reality because simulation is infinitely parallelizable, provides perfect repeatability, and has other great properties. It's also the safest place to build super-intelligence. Everything expressible through computing can be expressed in a Turing-complete simulation, including economies, language, mathematics, and science.
Silver believes the single most important thing is understanding the science of superintelligence through simulation. This allows understanding what it takes for super-intelligence to interact peacefully, including what conditions, structures, and levers ensure peaceful interaction between superintelligences and lesser intelligences. The goal is arriving at real science that helps understand the right things to do in the real world.
Ineffable has committed to limiting anything done in the real world to the availability of sufficient evidence from superintelligence science. They will not move to the next level in the real world unless absolutely confident it will lead to a good outcome. Silver personally believes the world with many highly intelligent and complex beings interacting will lead to a good outcome for humanity, with a balance of power where some support humanity's flourishing and protect against those that don't.
Silver acknowledges that monitoring agents' thought processes in English becomes increasingly difficult as agents become more powerful and may produce millions of words of thought or deliberately use deception. He suggests surveillance is still very useful but needs other elements beyond language-based monitoring. Ideas include using shadow actions to monitor what agents would do without actually allowing real-world actions, and using other superintelligence systems as monitors.
Silver strongly supports measured pace for real-world AI development while moving as quickly as possible in simulation. The reason is that simulation can develop the science of superintelligence that will help find solutions making things work well in the real world. He supports suggestions like those from Demis and Darian for slowing down real-world progress to allow governments, organizations, and emotional responses to adapt while enabling discovery of how to do things correctly through simulation and real science.
The journey will be extremely expensive in terms of computing, pushing the limits of what computing can do. Both Google and Nvidia were great partners. The company is not simply renting instant computing from the clouds as many other labs do, but is actually doing serious work on their own collection and other forms of computing that they own.
They are the first customers of the largest computing group in GCP, and by the end of the year it will start a month from now. They have a computing system that relies on graphics processing units (GPUs), specifically Nvidia units, which allows them to easily perform simulation, training, and deduction since these are largely general-purpose chips. The vehicle group has a structure where a single rack unit can be considered as one device, with very high bandwidth between nodes in a single rack, which is very important for reinforcement learning since they need very fast iteration cycles compared to large language models that are relatively slow.
They learned from work done 8 or 10 years ago at Google when there was an abundance of CPUs, but this is no longer expandable. They need to make sure that everything they do runs on GPUs and they don't need millions of central processing units alongside them. Their simulators can run on the GPU or for inference, training, etc. They have good infrastructure for that. Large language model companies are now facing a kind of problem that was faced many years ago, where they need isolated environments running on central processing units to simulate computers and so on. The goal is to avoid putting themselves in that situation.
Computing resources are divided into training, deduction, and simulation. The same computing resources are not equally good for training, inference, and simulation. GPUs have become increasingly dedicated to training over the past several years, especially since the advent of deep learning, and perhaps in the last few years for deduction, but less for simulation.
They want to be in a position where the rules of scaling matter, knowing that if they add a certain amount of computing and experimentation, they will get a certain amount of quality. Since GPUs aren't great for scaling up simulations, there are many different solutions including model-based reinforcement learning where the model becomes much more expensive compared to the simulation, and various approaches to data reuse or reuse of data multiple times. They are making a great effort to make the simulations as fast as possible and to ensure maximum utilization of the GPUs.
For reinforcement learning, the laws of measurement are slightly different because the problem is not fixed. As you progress in solving the problem, the data changes, or rather, the distribution of the data changes. There is some kind of measurement law where more computing brings you closer to achieving the goals, and this is something they are actively striving to achieve, especially as they move from a smaller computing cluster to a larger one.
Foundational work was done in computing and algorithms for reinforcement learning. When joining the deep learning team at DeepMind, most people were running reinforcement learning on a single CPU-based machine, so the range was very small. The work involved building a reinforcement learning agent that really takes advantage of GPUs and can scale linearly with computing. Thanks to its high efficiency, excellent ability to expand, and ease of modification and development, it became the most common reinforcement learning agent in the company, and at some point most reinforcement learning teams were using it or its derivatives.
Later when TPUs arrived, entire shelves of these units were available, so work expanded beyond that. Work at Google Brain after leaving DeepMind to establish the Google Brain office in Amsterdam involved expanding the scope to include sets of tensor processing units (TPUs), and that project was called "Seed RL." The goal was to make the best possible use of tensor processing units in both inference and training, while maintaining as much flexibility as possible in the environments, which can range from a game of "Go" to an expensive physics simulation.
Several of those algorithms have already been implemented, along with derivatives of these algorithms. Different algorithms are used depending on the environment, as there are advantages and disadvantages to each. There are some simple and fast-simulating environments, as well as other, more expensive environments. For anyone working today on large-scale reinforcement learning frameworks who has not read those original research papers, they are worth a look because many of the ideas currently in use go back to those early works.
The field of reinforcement learning is interesting to see now compared to what it was 10 years ago, or 8 years ago. The large language model community also uses reinforcement learning, particularly in the last two years or so. Many of the things they use are actually rediscovered techniques that have been used for many years.
The team has been intentionally kept relatively small, with 20 people hired in total. The approach is not to be in a hurry to build a giant corporation, as a lot can be accomplished nowadays with a small number of exceptionally talented people. For those hired, it is about finding exceptionally talented people, but it is not just about talent. It is also about bringing together people who are kind and cooperative as human beings, which is really important for the kind of culture being built. People who are perfectly aligned with the mission being pursued and who are somewhat suited to the needs are sought. The team is built with the right skill set as well. All four of these things are really important, with very high standards and expectations for the people who join.
Having such a relatively small number of people means there is a lot of computing power per person. These days many experiments can be conducted as just one person. It is important to maintain a very high ratio of computing power per individual so resources don't have to be fought over from day one.
Three of the co-founders worked together on the AlphaStar project, and many have already worked together. The AlphaStar team was an exceptional team that was deliberately assembled within DeepMind by selecting the very elite to work on that project. It is no surprise that four of the co-founders came from that team, but staff are not derived exclusively from that team or from DeepMind. The other three founders all came from very successful startups. Brilliant and pleasant people who will work well together, are aligned with the mission, and fit the needs are being hired.
A lot of effort is put into attracting the right people. Although it was difficult, many people worked with in the past had to be turned down simply because standards were too high. The process involved choosing from a very exceptional elite and assembling what is considered to be a team of stars. It is not easy to join, but the people brought in are amazing human beings.
People are drawn to work here especially at a time when the largest complexes and the biggest laboratories are caught in a "prisoner's dilemma" race with a lot of money at stake. For a group of people who truly believe deeply in the scientific mission, this must seem like a breath of fresh air.
A research-based culture has been established. It is not a company driven by the idea of building a subsequent incremental product like version 17.1 based on the above. It is a company built on researchers who pursue their passion and try to achieve the breakthroughs they believe are possible. The company has been focused around this from the beginning to create a research haven where the pressures of production are addressed, because achieving these breakthroughs will lead to much larger products later and greater successes than starting from day one with the approach of "this is the product we will do." It is research-based, creating a very special environment.
There will be some reasonable margins of uncertainty regarding predictions. Very strong evidence that the approach is on the right track will be seen within a year. Something that is true superintelligence according to the definition given earlier—anything that discovers things far beyond what humans can achieve in everything from science and mathematics and technology to languages, and so on—will be achieved. Initially the timeline was thought to be within 3 to 5 years, but timelines have accelerated a bit, so 3 years is predicted because of what can be done using programming agents to speed things up. Things will move quickly.
Keep Sequoia Capital in your library
Save the videos and channels worth coming back to, and find them again in one place.
