How Harvey Built a Research Lab on a Budget | Gabe Pereyra
In a Nutshell
Harvey's Gabe Pereyra shows how application-layer companies can compete with frontier labs by using open-source models, Neo Labs for post-training, and domain experts to generate synthetic legal data that bypasses customer data restrictions. The playbook starts with building realistic benchmarks like Legal Agent Bench and large diligence datasets, then moves through efficient RL environments, open-sourcing data for validation, and serving post-trained models alongside closed-source ones with proper evaluation infrastructure. Key insight: frontier ecosystem tools now make it possible to reach frontier-level intelligence on specific domains like legal work without the resources of big labs.
These notes were generated by AI and may contain inaccuracies.
Gabe Pereyra is co-founder and president of Harvey. He was previously a research scientist at DeepMind and then at Meta before showing his college roommates what GPT-3 could do. The duo then became Harvey.
The alternative title to this talk is building a research lab on a budget. It is an unfair game competing with the frontier labs if you are an application layer company. There are rich teams, poor teams, and then application layer companies. The frontier labs have more money, more talent, more compute infrastructure, and data.
The way to compete is by using the frontier ecosystem. When Harvey started 4 years ago, most of these companies either did not exist or were just getting started. Companies either had to build everything themselves or focus on something different like building their GTM org and a great product. Today using the frontier ecosystem you can compete with the frontier labs and build frontier intelligence.
The high-level playbook includes building benchmarks and training data, working with the Neo Labs to do post training, and serving models in production. To start, you want to build a benchmark. If you do not have a good benchmark, you cannot train models, and if you cannot train models, you do not need to serve them in production.
This year Harvey released three data sets. They started by building Legal Agent Bench, which is a taxonomy of tasks that associates would do at a large law firm. These cover multiple practice areas and include complex tasks like drafting complex fund formation documents and doing case law research.
Sign in to read the full notes
Get access to AI-generated notes, topic timestamps, and more.