Simulating Humans at Scale: Simile's Joon Sung Park
In a Nutshell
Joon Sung Park founded Simile to build large-scale simulations of human behavior using generative agents that combine LLMs with memory, planning, and reflection. Their core advance is a validation study showing these simulations predict real people's actions at 85% of human self-consistency accuracy, enabling companies like CVS to run concept tests, market research, and multi-agent scenario modeling without field experiments. Park argues this technology can extend beyond commercial use to solve fundamental social science questions, potentially transforming economics, policy, and collective decision-making.
These notes were generated by AI and may contain inaccuracies.
Joon Sung Park is inspired by science fiction depictions of technologically mature societies that feature two pillars: AGI and simulations that guide society. He sees an opportunity to build the simulation pillar today, a conviction built over years of research that he would not have held even five years ago.
Simile is an applied AI lab building simulations of human behavior and societies. Joon Sung Park is founder and CEO.
In April 2023 at Stanford, the team ran the Smallville project. The core observation was that large language models encode substantial human behavior from training data on the web and social media. When prompted with specific situations, models generate interesting micro-behaviors. This capability was the missing ingredient for complex agentic behavior.
Smallville created generative agents by pairing generative AI models with memory, planning, and reflection mechanisms. The simulation featured a game town with 25 agents. Each agent had a persona description and would wake up, follow routines, go to work, form relationships, and generate emergent phenomena such as parties.
The simulation was set the day before Valentine's Day. Isabella, a cafe owner, decided to host a Valentine's Day party, gathered materials, invited customers, and on the day of the event an immersive party formed with agents attending. Some invited agents forgot to attend. One agent named Claus, who received an invite, brought his crush on a date to the party.
Park's team had been excited about simulations and saw the vision of simulation failure early. His research career at Stanford began in 2020 around the time GPT-3 was emerging. He co-authored a paper on opportunities and risks of foundation models with Percy Liang and other Stanford researchers. While colleagues focused on classification and generation capabilities, Park was struck by the models' ability to encode human behavior.
Park comes from the social computing tradition within human-computer interaction. The core challenge in building social platforms is not UI/UX testing but understanding how tens, millions, or billions of people interact to create emergent phenomena, both positive and negative, and how to design for scale. Current practice relies on field testing by releasing prototypes, which is costly in human hours and risks negative outcomes such as emotionally harmful social media feed designs.
In 2022, the team published Social Simulacra, the precursor to the generative agents paper. The experiment simulated an entire subreddit by defining goals and moderation strategies, then populating it with thousands of personas. One simulation involved people discussing sightseeing places in Pittsburgh; personas collaborated to plan trips together.
After Social Simulacra, the next challenge was moving beyond simple personas to create complex agents capable of thinking over time to simulate longitudinal aspects of society, and validating simulation accuracy in practice.
Social Simulacra used GPT-3, which was janky, lacked instruction tuning, and required prompting tricks. The generative agents paper benefited from instruction tuning, enabling more complex reasoning about memory. Current foundation models have reached a point where these applications are imaginable.
Park notes that most large language model companies optimize for building super-intelligent, rational machines strong at technical problems with objective answers. However, humans are irrational, hold subjective values, preferences, and tastes. This creates divergence between moral science performance and the ability to predict and simulate human behavior. Current modeling paradigms have plateaued in simulating humans; the next frontier requires models geared toward modeling people's diversity.
The promise of applications was visible early through Social Simulacra. Park realized research excels at breadth while companies enable depth on a singular vision. Roughly six months after the generative agents paper, inbound interest arrived from social scientists wanting to run experiments and RCTs, followed by Fortune 500 companies and their leadership seeking to answer market research questions via simulation.
The team created simulations of one thousand people representing the US population. Using their architecture, the simulations predicted people's behaviors at 85% the accuracy of people replicating their own behavior. This result gave confidence to offer the platform for important real-world decisions. Co-founders include Percy Liang and Micah Bernstein; the three had worked together for five years at the time of founding.
CVS has partnered with Simile for nearly half a year. The buyer is a senior VP of human insights who read the validation paper and sought a solution to overcome limits on field-testable questions and the physics of human society. Connection occurred through a cousin who knew Park.
Customers typically work with polling and panel companies today. The Simile process begins similarly: customers identify populations of interest. Simile partners with vendors including Gallup to reach real humans and collect efficient, generalizable data. The goal is to identify the most valuable questions answerable in roughly 15 minutes of interaction. Collected data creates agents that answer questions far beyond the original domain. The platform is delivered as SaaS.
The approach parallels autonomous vehicle development: collect real-world data then augment with simulation to create models generalizable beyond training data. Here, the aim is to extract fundamental understanding of people that can be encoded into models.
Large language models are trained primarily on what people have said online. A significant gap exists between stated attitudes and actual behavior. Simile's simulations close this gap by collecting behavioral data and long-tail personal information, including life stories. Understanding where someone grew up and difficult life decisions provides a translational layer between attitudes and behavior.
Simile maintains a large repository of RCTs from social science and pricing studies. One training objective is building a foundation model of human behavior capable of predicting RCT results. Customers also explore responsible use of their internal data (for example, CVS's 90 million customers) to create fine-tuned, population-specific augmented models.
Data collection uses both interviews and surveys. Interviews capture long-tail information; one original study asked participants to tell the story of their life. Simile trains its own interviewer model via reinforcement learning to maximize information gained in minimum time. Surveys efficiently capture factual and discrete choice data on viewpoints, policies, and topics.
Simile builds its own models where gaps exist in encoding diversity of values, preferences, and tastes that rational models cannot capture. Current frontier models function like the CPU of intelligence—strong at objective questions. Simile aims to develop something closer to the GPU of intelligence: models that are as human as possible, with individual subunits representing real viewpoints of different populations. Frontier models are leveraged for research planning and coordination.
Customer engagements typically begin with concrete problems. Concept testing is common: customers test new product ideas or market messages and receive user reactions. The expanded vision includes testing thousands of ideas across thousands of subpopulations. Further use cases include product testing where agents experience a product for ten minutes and report observations, adding a temporal dimension. Multi-agent simulations are used for earnings call preparation, modeling how audiences would react.
Simulation offers advantages at scale compared to running Facebook ads for concept testing. The transcript cuts off mid-sentence while discussing the scale question.
The scale question is straightforward because experiments in simulation enable actual behavior simulation at scale. Any number of users can be pulled in without being bounded by the population available on Facebook. The approach is also much more representative because only certain groups of people respond to online experiments. One of the key promises of the model being created is that it is representative through the hard work of gaining a representative set of people and collecting data that properly represents them. Scale and representativeness are something many users do not have easy access to.
This is a common ask and pain point where the question many people have is not about what questions to ask but about how to get to the population they are excited to talk to in the first place, which is a huge bottleneck.
Downstream implications of decisions can be imagined beyond simply asking one question at a time. It is not just about whether someone likes a particular product or would pay for it. For a car company launching an electric vehicle, concept testing around marketing and the product can be done, but simulation also reveals what that does to the perception of non-electric vehicles, how it changes market perception, what that means for the rest of the product line, and how to balance second-order impacts of decisions in an evidence-based way.
There is no way to test for these second-order effects today, but they can be run in simulation. Going beyond asking one question at a time to think about the long-term implications of decisions is something customers are excited by.
Humans have a lot of randomness, so if asked the same question multiple times, the answer may vary slightly. There is a theoretical limit to prediction, but there are still a lot of gains in performance possible today. At the population level, the distribution of responses is measured using total variation distance, which shows how close the distributions of ground truth versus simulated information are. This metric is run across all customer use cases with a threshold believed to be good enough for decision-making. A TVD of less than 0.15 is considered quite strong evidence for making decisions. This is the northstar state for quantitative use cases and question-and-answer scenarios, and it covers RCTs which are many of the core use cases.
For multi-agent simulation and downstream implications, two categories of simulations exist: simulations that converge and simulations that diverge, and sometimes they coexist. The research question determines which category applies. For questions that converge, a little bit of error does not matter as long as the error is not so dramatic that it becomes completely detached from reality. The pull toward convergence is strong enough that the outcome can still be understood even if errors compound over time.
A good example is simulating a network of people where the network will always have a hub that gets formed, which network scientists call a scale-free network. This is what powered Google, as one of the core observations of PageRank was that regardless of how networks get formulated, some web pages get exponentially more links attached to them. This is a very fundamental human behavior also seen in simulated networks, and convergence always happens as long as human behavior is replicated with a certain threshold accuracy.
Questions that generally diverge include classical questions like whether World War I was inevitable. It can be difficult to run the same simulation over time and get the same exact outcome. For an election simulation, the same person may not win every time due to downstream implications of every single decision. The core evaluation for divergent simulations is around confidence. Running the simulation 100 times reveals how many times the results come out to be X, which can be used to create bootstrapped resampling to calculate confidence around the simulations. A huge part of the power of simulation is showing when it diverges to reveal the diversity of possible outcomes so people can understand the cause or mechanism of how those outcomes occurred and prepare for those futures.
The intuition is that an average function may converge while splitting outcomes to a binary may diverge. Technically this is a research topic where simulation is seen as a field akin to developing day one of inferential statistics. Inferential statisticians had to do a lot of discussion and research over time to decide that p less than 0.05 is evidence that is strong enough for science. The same kind of thresholds and standards are being set for the rest of the field. The right intuition is how to make a robust mathematical equation around when convergence or divergence will happen, and this remains a real research frontier for simulations.
A lot of Fortune 500 companies are coming to Simile. Non-existing corporate use cases might solve great mysteries of society. In economics, central bank decisions often reflect that macroeconomics is a function nobody fully understands, and oftentimes issues come from human psychology. Macroeconomics is a function of simulating human behavior at scale.
In the venture capital use case, internal debates about whether value accrues to a company could be addressed by running simulations of all the different layers of the AI stack to figure out where durability and value occurs. With a perfect simulator of human behavior, much more could be done than serving the Fortune 500.
When researching this area, advisors Michael and Percy were told there is a Nobel Prize to be won, and this is truly believed. Classical economics simulations like agent-based models pioneered understanding of topics such as how segregation happens and the causal mechanism for segregation. Scholars like Thomas Schelling built agent-based models that were extremely simple and rudimentary but showed something deep about human macro behaviors and went on to win a Nobel Prize.
The same opportunity exists in an augmented way. Previously, agent-based models were deterministic where in a simulation of the motor of segregation, an individual agent was simply a red dot or blue dot, and in every game iteration they would look around their corner, see how many neighbors were the same color, and if that threshold went below a certain threshold they would move to a new location. Now real agents can be created that replicate the full richness of individuals and run the same kind of simulations.
Questions that go beyond commercial use cases include, in the context of macroeconomics, when bank runs happen, climate change where the core blocker is the collective action problem of many nations, signals of a democracy about to collapse, and the origin story of the monetary system. These simulations are believed to be the northstar state of this field. These would involve very large scale simulations with many agents interacting with each other.
A future is seen where a simulation takes $100 million to run once and could take many months to run, but when run it solves one of the fundamental questions of society. This is a genuinely exciting possibility for the field.
Politics could be forever changed if everyone with an agenda about how a policy change will impact things could run the simulation and understand all the downstream implications, not just what is going to happen this year but what it means in the next 5 to 10 years.
Science fiction covering societies that have progressed far enough in technological maturity always shows two pillars: some version of AGI and some version of simulations that really help guide the society. There is an opportunity today to take the first crack at building the simulation. This would not have been said even five years ago, but conviction has been built over the years going deep into this research.
There is a clear use case today that can serve users, but there is a lot of innovation yet to come that will build up to actually building a simulator akin to the CERN of human society. One of the things co-founder Percy sometimes says is that the greatest scientific innovations often start from an amazing measurement. The Hubble telescope really changed the trajectory of how we understand the universe. Simulation can be that for human society.
There is a lot of focus on natural sciences, but the question is how simulation can really unlock understanding of humanity and social sciences and how it can be used to make society a better place. There is a small but breathtaking chance that the field of economics as it is known may actually become solved by simulation, and this extends not just to economics but to everything that deals with human behavior and social sciences, which ultimately is everything around us.
Keep Sequoia Capital in your library
Save the videos and channels worth coming back to, and find them again in one place.
