Inside the Robot Data Factory: Mecka's Josh Gao on Sensorizing Humans to Train Robots
In a Nutshell
Mika AI is building the infrastructure to collect and process millions of hours of human sensor data—using wearable devices like force-sensing gloves and multi-camera rigs—to train robotics foundation models, addressing the critical data bottleneck that separates robotics from the internet-scale advantages of language models. The company operates a dual-sided data factory combining consumer apps for household tasks with B2B partnerships across industrial settings, while developing hardware to capture force, tactile, and egocentric video data that video alone cannot provide. Success depends on gradually opening commercial use cases that fund data collection, moving from specialized industrial robots toward general-purpose humanoid systems through a "data soup" approach combining human demonstrations, remote operation, and eventual simulation-based self-play.
These notes were generated by AI and may contain inaccuracies.
There is an idea in science fiction that suggests that when robots and artificial intelligence eventually achieve full generalization and solve all the tasks that humans can do, the ultimate frontier will be organic. The final boundaries will be biological. This concept appears in Asimov's works, in "Ender's Game," and in "The Hitchhiker's Guide to the Galaxy" at the end, where the planet was actually a giant organic computer trying to process and understand the meaning of life or what the question is.
Josh Gao is the founder and CEO of Mika AI, which is building a data and publishing company in the field of robotics. The host and the entire system believe that robots are about to enter a golden age. Unlike large language models, where there was lots of free human data from the internet, robots are essentially data-constrained.
Mika AI's position in the ecosystem as the leading data factory for bringing robotics and physical data to be distributed to models is considered very interesting. The conversation focuses on how environmentally focused data and data collection work in robotics, what the actual processes and economics are of running a data factory, and general thoughts on robotics research and dissemination.
Mika is a data and publishing company for physical artificial intelligence. Today they are creating, processing, and licensing a lot of data that goes into training foundational robotics models. The big problem in robotics is that there is no luxury of a cold start that large language models had with the internet. Much of robotics data consists of three things: the pixels, the actions, and the logic. There is no access to a large amount of organized, processed, and clean data to train models to understand what existence is like in the physical world.
The work focuses on: First, creating different types of devices such as sensors and wearable devices for humans; The various processes for collecting data and partnering with individual and corporate stakeholders to carry out tasks across different environments; Then finally, post-processing and quality gate control to make these datasets useful and teach robots about the world.
The question arises whether robots, with the added burden of paying for data collection and provision, represent a feasible challenge and if it is economically viable without the gift provided by the internet. The response indicates that there is still the luxury of having many of the things that the internet has given. Pre-training data in laboratories has always relied heavily on web crawlers and the like, which didn't become a big market, but today in robotics, pre-training is seen as a very large market.
This can take many forms, including teaching video models how to generate a more physics-based reality of what the world looks like. All the work done in vision and language models and various types of previous world models can be leveraged. Whether it is possible and scalable to access that volume of data ultimately comes down to the commercial use cases that can be opened up at different levels of data sizes.
The Mika website states they are teaching robots about carbon-based life. The phrase was created the night before the first announcement on Twitter. The science fiction idea suggests that when robots and artificial intelligence completely generalize their tasks and solve all the tasks that humans can do, the ultimate frontier will be organic. The final boundaries will be biological, as seen in Asimov, in "Ender's Game," and even at the end of "The Hitchhiker's Guide to the Galaxy," where the planet was ultimately a giant organic computer trying to process and understand the meaning of life or what the fundamental question is.
The human brain only operates on 20 watts of power and possesses the ability to self-heal. In the journey of teaching robots and developing their ability to generalize fully, they will slowly show them what carbon-based life is made of.
Josh Gao comes from an unconventional background in robotics, having started in financial technology and the world of athletic shoes. Prior to this, he was building a Toast-like system for cloud kitchens, managing orders in back kitchens. During his fourth year of college, he would skip lectures to set up kiosks in rural areas of Rochester, New York, staying up all night replacing restaurant storefronts.
His co-founder built a stablecoin that was eventually sold to Coinbase and has a very obsessive personality, being among the top thousand players in four competitive games including "League of Legends." What brought them together was being Canadians and having a strong background in science fiction. The ultimate belief was that payments were a difficult and complicated problem, but they wanted their next job to be something exciting that would make the future look amazing. The field of robotics seemed like a great place to start.
The advantage of entering the field without a robotics background was having to do a complete exploration process. Hundreds of people in the field were contacted, every research paper available was read, and several robotics labs were visited. Around the end of 2024, GPT technology was taking off strongly while robotics was still evolving from the era of traditional robotics. It seemed clear that data and computing would become very important in this industry.
Research papers such as Ego4D were read from a new perspective, noting about 4,000 hours of human illustration data. As the data was delved deeper into, it was realized that some datasets were for people's everyday lives such as bowling, with very wide-ranging use cases and potential applications across the 4000-hour range. A first-principles approach was used to form a general intuition about what would be important.
Egocentric data is simply video data of people performing and displaying tasks from a first-person perspective. The general idea was that by bridging the gap between human data and robot data, a large part of the visibility gap could be reduced. Since robots experience the world from their point of view, showing them what it looks like to interact with the world from a first-person perspective was considered valuable.
The bet on egocentric data was made because when the field was first explored, much of the robotics data at that time was remotely operated, where humans controlled robots to provide demonstrations. It was very difficult to expand the scope of that at the time. Through GPT and many of the scaling principles, ways had to be found to increase the data size much more.
The early intuition was that humans and robots share many similarities. Humanoid robots had just begun to take off, and humans and humanoid robots look similar. The question was why humans couldn't be transformed into this kind of collective learning that Tesla has, so that everyone could contribute to understanding and establishing the physics of the world.
When first starting, the first pilot project was to create a mobile application where Chinese aunts and uncles would film themselves cooking, cleaning, and performing their daily tasks. This has developed significantly to encompass all the specialized devices now produced and all the different partnerships needed across different sectors. Ultimately, sensors are being integrated into humans.
Force data is extremely important because in autonomous driving, power data is not needed as much, whereas in robots, especially in handling objects, it really matters how hard a doorknob is squeezed or when picking up an egg. Video does not capture that accurately. Some of the force data alternatives seen are approaches like wrist cameras currently used with humans and robots, and the ability to see the contact time between the hand and various objects interacted with. This is clearly not an ideal way to get an understanding of how much force in Newtons is required to pull a chair or turn a bottle cap.
Power data is one of the products being worked on building. When dealing with many robotics fundamentals labs, it is realized that to bridge this gap, new devices must first be invented. Things like gloves that humans wear were developed, and through those gloves information about how much strength is required to perform daily life tasks can be gathered.
There are many different approaches from the operators' side. One is the direct, consumer-oriented approach through the mobile app, where anyone can sign up from anywhere in the world, start contributing, register their occupation, and receive a series of scheduled tasks to do such as cleaning their bedroom, preparing their breakfast, or cleaning up afterward. This is a very simple method for getting diversity in pre-training data.
The second approach involves collecting a lot outside the home, requiring the building of a business-to-business engine and developing the ability to recruit companies across the categorization of all physical human labor, from bakeries to circuit board manufacturing lines, and everything up to voice recording.
The actual data engine of the data factory is far more complex and much more difficult than people might expect. Many of the things distributed in the world did not exist until designed and produced, with touchscreen gloves being a great example. There is nothing off-the-shelf on the market that can be bought and worn that reliably tells how much force is being applied day after day and converts that into ready-made datasets for basic robot models.
There needs to be a hardware iteration loop with three or four different production cycles, and the engineering ability to think and develop what will be useful, then scale that production up into the world. This turns into distribution logistics for thousands of homes and thousands of businesses to collect and lift the data. 100-gigabyte data upload lines had to be set up in many places where operations occur just to support the hundreds of petabytes of data that must be dealt with. Then the data must be processed, quality gates must be ensured, and computer vision techniques must be used to make them useful.
Quality issues in robot data collection mean that many hours of robot data sample submissions are ultimately not accepted. The lesson originally came from different types of robot data, where huge robot factories were set up in large warehouses performing repetitive tasks to try to expand the range of remote operating data. Research indicated there is a certain limit to diversity where it becomes saturated with robotics data or data in general.
With human data, quality is a somewhat moving target as growth occurs. The number of exceptional cases encountered in the real world that must be dealt with is extremely difficult, and something new that has never been seen before will always be found. The focus is on thinking about inference algorithms capable of detecting many different things while processing them, and having different engines for escalation at very high speed.
An example of something monitored is fraud, which many people underestimate in the data industries. Since data that does not yet exist is being created rather than simply filtering data from the internet, creative and interesting things are encountered. At one point, individuals were recording YouTube videos in a first-person perspective while working at McDonald's and trying to present that as their contribution.
This was part of the painful lessons where many verification gateways had to be created, but ultimately presence where the collection process is taking place is necessary. How to do that must be seen, and an incentive model that actually motivates people to make a good contribution must be created.
The shareholder's share in terms of margin cost is not actually the biggest factor to ensure the end-to-end usefulness of this data. Although the foundation will always exist and perhaps become more competitive, there will be ways to differentiate and ensure that data is of very high quality and verifiable on a very large scale using the appropriate hardware, processes, and software.
When thinking about these products, the goal is to expand and develop them, and like any product that starts to expand, improvement is wanted. Over time, different products may be evolving. When thinking about egocentric podcast data as one storage unit, it is realized that different kinds of data will become important, such as moving from a roadside Chinese restaurant to Michelin-starred restaurants over time, or getting closer to what it means to be a Federer compared to being an average tennis player. These products will change, and the way they are created will also change.
Millions of hours of data have been reached, representing a huge leap from Ego4D's 4000 hours two years ago. By comparison, if thinking of self-driving as an example, Tesla has spent more than a decade solving two degrees of freedom of movement, acceleration and direction, with millions of cars on the road, and now has far more than a few million hours of video data. Relatively speaking, many orders of magnitude away from where things should eventually be.
On one hand, this looks like an opportunity with a lot of revenue for robotics data factories. On the other hand, as someone investing in foundational models of robots, the calculations show that a car has two or three degrees of freedom of movement, whereas a robot has dozens in one hand. When combining that with the potential need for force data, there is a lot to it in a much more difficult environment to work in.
The conclusion might be that billions, trillions, quadrillions of hours might be needed. The question is whether there is enough global GDP to support the cost of bringing that in. The opinion is that it is about gradually opening up commercial use cases, so it shouldn't be so binary, meaning one day the world of robots will be opened up or it won't. It will be a slow creep into the world.
One great way to get there is to build part of this basic pyramid, where generalizations are seen and the laws of measurement in action give people the courage to expand. With that courage, some ability to generalize in some of these environments can be gained, and a lot of data can be collected within the scope and task to bridge this training gap. Data can perhaps be collected directly from a room, specifically in a podcast presentation, to bridge the gap in what in many cases amounts to four-tenths of the reliability in real-world environments.
The most abundant data currently seems to be household data, simply because everyone can film that and it's very easy to get. But the home seems to be one of the most difficult environments to deploy robots in. The home will be among the environments where the most care must be taken of all. Many of these standards and assessments, which are just beginning to mature across the robotics field, ensure that the robot does not fall on the child at home, for example.
The initial tasks will become much more restricted as progress is made in this range, so that they are very goal-oriented, where some of these tasks can be improved, and where there are standard operating procedures for what it means to do this job and this work. Like a laundry folding robot, which has been worked on for over a year and a half trying to solve the laundry folding problem, and it is believed it will definitely be solved in the near future. Many of these pick-up and placement tasks, and packaging tasks seen on assembly lines that required handling robots that could not be solved by traditional robots, will be reached within the next year or two.
Useful robots can already be deployed today because people underestimate how diverse robots are. If limiting to humanoid robots, they are very popular and would also be a very common end state for robots. But apart from that, there are many different forms that exist in the world of robotics today. Different markets can be looked at, even in hotels in China or Southeast Asia, for example, where there are robots that provide services, do cleaning, and deliver orders after the Uber Eats service arrives at the door, from the reception desk to hotel room doors. Many of these robots are widely deployed throughout the world today and have just begun to spread in the United States and North America.
Some of the first robots that should be deployed will even be just industrial cleaning robots, for example. There are many themed robots currently available. Then slow progress from there occurs, starting to solve basic handling tasks along that range, all the way to those humanoid robots that will directly handle any task in the future.
When reaching the point similar to FSD's flywheel system, the transition will be slow. Business tasks will gradually enable closed loops with actual operating data including failures and debug data. This data will be extremely important in launching closed robotics loops. The timeline depends on covering different types of tasks and areas that haven't yet been resolved.
Miko will look very different during this process. The company will not rely entirely on self-centered data, believing that diverse units and complete data mix are important. Future possibilities include producing high-quality "Michelin Star" data, teaching robots real-world exceptions they haven't encountered, and contributing significantly to robot deployment by collecting data for specific areas, providing follow-up training, and helping support companies.
Drawing an analogy to LLM dissemination patterns, three vectors exist: vertical application companies specializing in specific areas, foundational model companies expanding market reach, and service providers helping companies adopt AI at scale. In robotics, both vertical and horizontal approaches will exist. Vertical sectors will handle specific tasks from devices to models and data, while an ecosystem will feature diverse robot shapes and sizes with different open models.
The future will resemble modern robotic versions of systems integration companies. Unlike traditional installation with narrow profit margins, the new model involves installing robots and making them more data-driven and productive over time. The goal is enabling robots to perform tasks impossible days after installation and helping businesses expand their operational scope.
Industrial robots represent the best starting point since they possess traditional robotic technologies and operate in limited, specific tasks like data center maintenance where environments are more restricted than open outdoor settings. Early-stage viable categories include hotel delivery systems from reception to guest rooms and environments humans cannot reach, such as asteroid mining operations.
Gecko Robotics performs ship maintenance and similar tasks in inaccessible locations, representing the type of specialized applications that will drive early adoption.
Mobile app development enabled rapid scaling but lacked rich world information. Early development focused on "turning humans into Weemos" by equipping people with sensors. Products include baseball caps with various cameras and sensors, a central hub connecting accessories like touchscreen gloves and outdoor cameras, all designed to capture daily human-world interactions for behavior replication.
EgoVerse was initiated by Professor Dan Fay and Ryan Semer as early believers in self-centered data. Contributors have provided diverse self-centered data across companies, industries, and home environments, sharing traffic path data including post-processing, computer vision, and quality gateways, plus platform and web tools, data loading, and infrastructure contributions.
Jim Vann presented a visual representation at a Sequoia event showing the convergence toward a data mix including internet-sized YouTube data, self-centered data, tele-UMI data, and remote operation. The industry has moved from debates about single data types succeeding to recognizing all data as essential "data soup" ingredients. Current consensus includes approximately a million hours of human data through self-centered vision and media, tens of thousands of hours of UMI data, remote operation data for final stages, and in-band and distributed data within demonstration hours.
Tactile data has not yet entered the mix but will be necessary if models need to learn force application, such as turning doorknobs. Simulation data will also become important for converging toward unified goals.
The tennis learning objection parallels AlphaGo development: initial large-scale behavioral cloning of expert play, followed by self-play within game rules leading to superhuman moves like AlphaGo's move 37. In robotics, the "rules of the game" encompass the entire physical world scope, requiring self-centered data to establish basic physics and behavioral replication while shaping future world simulators.
Future systems will likely combine self-play in both physical reality and simulation, enabling scenarios like dropping cups without actual destruction while maintaining real-world grounding.
World models (used interchangeably with video models) offer value in enabling self-play scenarios and translating internet/YouTube pixel prediction work into procedural paths for acceleration. The current data pyramid will expand through additional methods rather than simple scaling.
The GBT-3 moment analogy for robotics is no longer preferred. Simple scaling of current data mix ratios will solve many tasks in restrictive environments with specific standard operating procedures, but achieving general-purpose robots capable of folding clothes or cooking meals requires new algorithmic and data breakthroughs beyond current approaches.
Future development will introduce touch data and move toward omnimodel approaches capable of accepting any input type (force, pixels, potentially smell) and generating corresponding outputs (action, sound, control signals). Olfactory sensing remains in early development stages.
GBT-6 Astra results show strong generalization to robotics tasks. Foundation labs possess courage to expand data scope with imperfect information. Adding robotics data to pre-training sets enables cross-domain generalization. Multi-model approaches will allow different observation types (force, motion paths) while specialized models may achieve higher reliability for specific tasks than generic all-purpose models.
The company began with a childhood vision of robots everywhere. The future will feature strange robots performing unimagined tasks like asteroid mining, embracing the ridiculousness exemplified by "Kawasaki Carlio" - a robot combining cheetah, dog, and horse characteristics featured in absurd advertisements showing fictional riders jumping rivers.
Robot numbers will exceed human population many times over within a decade, potentially reaching 100 billion units with diverse forms including "tiger-dog-horse" robots.
Keep Sequoia Capital in your library
Save the videos and channels worth coming back to, and find them again in one place.
