[AI Gets a Body] 04 : Data Is the Bottleneck
Teleoperation data, synthetic data, world models, and the factories that stamp out demonstrations
Language models started from the internet, a vast warehouse. Robots have no such warehouse.
In Part 3 we saw three ways robots learn. Whether they imitate people, learn by trial and error, or practice in a virtual world, they share one need: material to learn from, in other words data. Text and images were already piled up on the internet. A record of a robot grasping and moving objects has to be made by someone actually moving a robot. This part follows who collects that material, how, and how much, using figures from the papers.
Redrawn by the author from the data pyramid described in NVIDIA's GR00T N1 paper (Fig. 1).
Why it is a bottleneck
A language model’s raw material is text people already wrote. What a robot needs is a record of “what the joints did in response to what was seen.” That record is created one line at a time, each time a robot moves. This is the author’s own explanation. That is why robotics research has studied the way data is collected as a subject in itself, and the approaches fall into four groups: pooling data across institutions, collecting directly in the field, building a dedicated facility, and generating it by computer.
Path one: pool it
Open X-Embodiment answered the problem with “let’s share.” According to the paper’s abstract, 21 institutions collaborated to combine data from 22 different robots, covering 527 skills (160,266 tasks). The key point is that the robots’ bodies differ. The authors report that an RT-X model trained on this data improves the capabilities of multiple robots by drawing on experience from other platforms. Another robot’s experience can help yours. The abstract, however, gives no numbers for this effect.
DROID goes a different way. It aimed to capture the diversity of real-world settings, not the lab. According to the abstract, 50 data collectors across North America, Asia and Europe gathered 76k demonstration trajectories, or 350 hours of interaction, over 12 months across 564 scenes and 84 tasks. The authors report that training with DROID yields policies with higher performance and improved generalization.
Path two: build a facility
AgiBot World built the collecting factory itself. The paper describes a 4,000-square-meter facility with 100 robots collecting data. The latest version holds 1,001,552 trajectories, 2,976.4 hours in total, 217 tasks and 106 scenes, across five domains: domestic, retail, industrial, restaurant and office. Skilled operators control robots with VR gear or whole-body motion capture, and after collection, annotators check each episode against the standards. The paper calls this procedure human-in-the-loop verification.
The weight of that verification shows in the paper’s comparison. On one task (wiping a table), 528 human-verified trajectories gave a completion score 0.18 higher than 482 unverified ones. The paper notes that more data does not necessarily mean better performance. The implication is that quality control drives data cost as much as volume does. That interpretation is the author’s.
The AgiBot team reports that policies pre-trained on their data achieved an average performance improvement of 30% over those trained on Open X-Embodiment. All evaluations were run in real-world settings with 30 trials per task on average. Still, this is the authors’ comparison against their own dataset, a self-report and not an independent verification, and it is worth keeping in mind.
Path three: collect without a robot
If you have to buy robots and station people beside them, scaling is hard. UMI (Universal Manipulation Interface) collects human demonstrations with a hand-held gripper. The paper’s abstract describes the approach as portable and low-cost, and says the learned policies are not tied to one robot and can be deployed across multiple robot platforms. The same paper reports that policies trained on diverse demonstrations generalize zero-shot to new environments and objects. This path differs from the previous two in that data comes straight from a person’s hand, without passing through a robot.
Path four: make it up
The last path is to have the computer produce the data. NVIDIA’s GR00T N1 paper organizes it as a pyramid. At the base sit web data and human video, in the middle synthetic data from physics simulation or neural models, and at the top real robot data.
The scale the paper reports is striking. It generated 780,000 simulation trajectories, equivalent to 6,500 hours (nine continuous months) of human demonstration data, in 11 hours. On the video-generation side, starting from 88 hours of real teleoperation data, it produced about 827 hours of video, roughly a tenfold increase. That video took about 105k L40 GPU-hours (around 1.5 days on 3,600 GPUs), and on one GPU a single one-second video took 2 minutes. All of these are figures NVIDIA reports about its own model.
Values reported in each paper (DROID, GR00T N1, AgiBot World). Synthetic "hours" are converted to equivalent human demonstration time and are not the same as real collection time.
One caution in reading the figure. The 6,500 simulated hours are not time a robot spent moving in reality. They are an equivalent amount in human demonstrations, and the gap with reality leads back to the sim-to-real problem from Part 3. The 827 hours made by the video-generation model are likewise generated footage. That equal bar lengths do not carry equal weight is the author’s interpretation.
Where world models stand
World models sit at the center of synthetic data. NVIDIA’s Cosmos paper states in its abstract that “Physical AI needs to be trained digitally first,” and presents a platform that bundles a video curation pipeline, pre-trained world foundation models, examples of post-training, and video tokenizers. The models are said to be released as open-weight under permissive licenses. How convincingly a world model imitates reality remains a question that needs continued testing. We could not confirm that degree as a single figure in these papers, so only the structure is introduced here.
Seen through a business lens
The four paths have different cost structures. Pooling costs collaboration, field collection costs people’s time, a facility costs equipment and operations, and synthesis costs GPUs. This split is the author’s own summary.
What the papers confirm is this. Facility-scale real collection produced a million trajectories, while synthesis produced far more volume in far less time. But synthesis leaves the homework of the gap with reality, and real collection demands quality control through verification. As the author’s own estimate, a robot company’s moat may come less from the amount of data than from a pipeline that makes data cheaply and accurately. From a business view, selling one more robot and accumulating one more hour of data compete inside the same investment. This is an interpretation, not something the papers state.
One more point about sources. Every number above was reported directly by the researchers who built the dataset or model. They come from different robots, tasks and evaluation methods, so the bar lengths should not simply be ranked against each other.
Seen from the market: what data costs and how big forecasts say it gets
What it costs to collect data varies enormously by method. The UMI paper states that the 3D-printed gripper has a bill-of-materials cost of $73, and the GoPro camera and accessories total $298. Adding them gives $371, which is the author’s arithmetic. The paper notes that teleoperation requires high setup costs for hardware and expert operators. The authors’ project page reports demonstrations collected per hour on a cup-arrangement task: 111 with the UMI gripper, 231 with a human hand, and 35 with Space Mouse teleoperation, over three times faster. That speed comparison comes from one task, measured by the researchers. That figure does not include a robot arm (the paper gives no arm cost), and the two methods yield different kinds of data, so it cannot be compared plainly with teleoperation equipment. Neither paper contains a dollar-for-dollar comparison of collection cost.
Market forecasts span a wide range by institution. In a February 2024 article, Goldman Sachs Research projected the humanoid robot market could reach $38 billion by 2035 (1.4 million units shipped). That raised an earlier $6 billion projection. The article says the manufacturing cost range per unit moved from $50,000–$250,000 the year before to $30,000–$150,000. In a May 2025 article, Morgan Stanley Research said the market, counting supply chains and repair, maintenance and support, could surpass $5 trillion by 2050. It assumes the price in high-income countries falls from about $200,000 in 2024 to about $150,000 by 2028 and $50,000 by 2050. The two forecasts differ in horizon (2035 versus 2050) and scope (the robot market versus supply chain and services included), so they cannot be swapped for one another. Neither institution lists data supply as an explicit variable; that is the author’s reading of the two articles. What is clear is that both forecasts assume falling unit prices. That prices fall only if robots become useful, and that data is the key to that, is this series’ working hypothesis.
Funding flows have little that can be confirmed from primary sources. There are press reports that Physical Intelligence, a robot foundation-model company, raised a large round, but we could not find a post in which the company itself states the amount or valuation, so no figures are given. Its homepage says only that it is grateful for the support of Bond, Jeff Bezos, Khosla Ventures, Lux Capital, OpenAI, Redpoint Ventures, Sequoia Capital, CapitalG and Thrive Capital.
Next time
Part 5 looks at the limits of the body. We examine why actuators and reducers, hands and tactile sensors, batteries and on-device compute matter as much as the brain.
- Open X-Embodiment Collaboration, “Open X-Embodiment: Robotic Learning Datasets and RT-X Models”, 2023. arXiv:2310.08864
- Khazatsky et al., “DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset”, 2024. arXiv:2403.12945
- AgiBot-World-Contributors, “AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems”, 2025. arXiv:2503.06669
- Chi et al., “Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots”, 2024. arXiv:2402.10329
- NVIDIA, “GR00T N1: An Open Foundation Model for Generalist Humanoid Robots”, 2025. arXiv:2503.14734
- NVIDIA, “Cosmos World Foundation Model Platform for Physical AI”, 2025. arXiv:2501.03575
- Goldman Sachs Research, “The global market for humanoid robots could reach $38 billion by 2035”, 2024. goldmansachs.com
- Morgan Stanley Research, “Humanoids: A $5 Trillion Market”, 2025. morganstanley.com
- UMI project page(data collection speed comparison). umi-gripper.github.io
- Physical Intelligence homepage(investor acknowledgment). pi.website