Safeworld's simulated humans test whether AI-driven robots will stop in time

Kyle Wong, one of Safeworld's co-founders, describes the hazard in terms a plant safety manager would recognize. "One of the most common areas is if there is a blind corner in this particular factory," he said. "What is the speed or what is the stopping distance that you need to make sure that this robot will not collide with a particular human? If a human is carrying boxes, for example, will the robot detect the human or not?" ^[7]

With a generative AI model driving the robot, that question is hard to settle on paper. TechCrunch's report says the architecture is not predictable the way traditional algorithms are ^[3]. Vishal Dugar, CTO of Gritt Robotics, described the same problem from the builder's side. "The difficulty with most of our systems is it's very hard to formally prove it by doing some math, writing some equations, and saying yeah, the system is verified to be safe," he said. "It necessarily has to be done empirically." ^[11]

Ding Zhao's worry is the gap between the demo and the site. "It is not the robot in the vacuum, in the demo, that we are worried about," he said. "It is the robot that is deployed at scale, with people who potentially never operated a robot before." ^[16] Here's what people on a site actually do, by Dugar's account: "They could be kneeling, standing. They could be tripping and falling potentially. They could be crouching. They could be running." ^[13] Simulation lets a team test the fall without staging one. "Otherwise, you would have to go and trip and fall for the robot, which is like a hard thing to be doing all the time," Wong said. ^[8]

The thing being pitched is larger than a simulator. Zhao frames the work as two problems, the first being "how do you underwrite the risk of a probabilistic system?" ^[10] "The second part that's really hard is the trust part, and you need both to deploy a robot," he said. ^[12] Jonathan Lai, the a16z Speedrun partner, framed the investment around a standard. "The time to build an industry safety standard is now while robots are being designed and deployed," he told TechCrunch. "By the time you have robots in households colliding with kids and causing safety incidents, that's way too late." ^[14]

The thing being done today is narrower. According to TechCrunch, the platform resembles tools robot builders already use internally, and the founders argue that makers will still want someone outside the company to check their work, even if the only reason is to let competitors swap what they know about safety cases ^[15]. Gritt, whose robots help workers install solar panels at industrial-scale solar farms, is partnering with Safeworld as it develops its simulations ^[9]. Lai's example was a home with children ^[14]. Zhao's "underwrite" is insurance vocabulary, but the report does not quote an insurer, regulator or site operator who will require an outside safety case, and it does not include pricing ^[10].

Two questions place a robot program on a grid. The first is whether its behavior can be shown safe on paper or only tested empirically, as Dugar says of his own systems ^[11]. The second is whether anyone outside the maker has to accept the evidence. Provable behavior with an internal audience needs only the reviews a team already runs. Add an outside reviewer and it becomes a documentation job. Empirical behavior with an internal audience can stay on in-house simulation; by the founders' own account, those tools resemble Safeworld's ^[15]. Empirical behavior that has to convince a customer or a site operator is the buyer Safeworld's seed round is meant to find ^[1].