Designing the seams in physical AI systems

Physical AI is hard to design because intelligence is only one part of the product. A deployed system is a chain of handoffs: sensor to model, model to controller, controller to actuator, mechanism to enclosure, and software release to hardware revision. Each handoff carries latency, uncertainty, tolerances, and distinct failure modes. Controlled demos can minimize these seams. Products have to manage them, which makes design a systems discipline as much as a machine learning discipline.

The useful unit of design is therefore not only the learned policy. It is also the boundary between that policy and the rest of the system. This essay focuses on those boundaries and on the role of industrial, interaction, and systems design in making them explicit, testable, and maintainable.

The seam between thinking and moving

Current physical AI architectures make the first seam visible. Figure's Helix, which controls the upper body of its humanoid, is one end-to-end vision-language-action neural network with two components trained jointly to communicate. System 2 is a 7-billion-parameter vision-language component that processes the scene and instruction at 7 to 9 Hz. System 1 is an 80-million-parameter visuomotor transformer that turns the latest semantic representation into continuous control for the wrists, fingers, torso, and head at 200 Hz. Figure states the tradeoff plainly: "VLM backbones are general, but not fast, and robot visuomotor policies are fast but not general." In deployment, the two components operate asynchronously on separate embedded GPUs, allowing semantic intent and reactive control to update at different rates within the same network.

Physical Intelligence's π0 addresses the same timing requirement with another vision-language-action architecture. It starts with a 3-billion-parameter vision-language model and adds continuous action outputs through flow matching, a variant of diffusion, to produce motor commands up to 50 times per second. Its training mixture includes data collected across eight distinct robots. Physical Intelligence motivates the work through Moravec's paradox: tasks such as folding a shirt or clearing a table can present some of AI's hardest engineering problems.

These architectures expose a design choice beyond model selection: how general semantic processing and fast control divide the work. The contract between them needs to specify update rates, representations, acceptable staleness, fault behavior, and ownership of safety constraints. Those details may be hard to see in a demo, but they determine how a robot responds when the scene changes between high-level updates.

The seam between simulation and the floor

Collecting real robot data at foundation-model scale is slow and expensive, so many development pipelines combine physical data with simulation and other sources. NVIDIA describes this as three computers: one for training, one for simulation and synthetic data generation, and one for inference on the robot. Its Isaac GR00T humanoid models use "real captured data, synthetic data, and internet-scale video data." Its Cosmos world foundation models can generate possible futures from text, image, video, and action inputs and support what NVIDIA describes as physics-based simulation for closed-loop policy evaluation.

Researchers call the resulting boundary the reality gap: "the collection of discrepancies, from physics to perception, that can cause a policy to fail when transferred from simulation to a real robot." The cited survey groups the sources into dynamics, perception and sensing, actuation and control, and system design. That last category includes communication delays, safety mechanisms, and differences in the software stack. Simulation often omits or simplifies network jitter, thermal throttling, watchdog behavior, sensor degradation, and similar effects unless they are explicitly modeled. Hardware-in-the-loop testing can cover some interactions between software and physical components; the rest require testing on the robot and in its operating environment.

The survey sorts responses into two strategies: reduce the gap through methods such as system identification, learned residual dynamics, and hardware co-design, or overcome it through methods such as domain randomization, fine-tuning on real data, and online adaptation. The choice should be made per subsystem rather than deferred as a general research problem. A simulation plan should state what a simulator is expected to verify, what requires hardware-in-the-loop or floor testing, and how much discrepancy the system can tolerate.

Latency is a safety budget

In consumer software, latency is often an experience metric. In physical AI, it can change the space required for safe operation. Speed and separation monitoring, as specified in ISO/TS 15066, computes a minimum protective distance between a person and a robot. Published implementation work explains the terms: the person's approach speed, the robot's motion during reaction and braking, the time required to detect the person and stop, and margins for intrusion and position uncertainty. The paper also recommends a control-loop update rate of at least 100 Hz for smooth proximity-based speed adjustment in the implementation it evaluates.

The distance added by a delay is the relative approach speed multiplied by that delay. At a relative speed of 2 m/s, for example, 1 ms corresponds to 2 mm of travel, while 100 ms corresponds to 20 cm. The total protective distance must also account for braking and uncertainty. A heavier arm or payload can increase stopping time, although braking performance also depends on speed, pose, controller behavior, and brake condition. Mass, compute, thermals, sensing, control latency, and enclosure geometry therefore contribute to one system-level safety budget rather than independent model and mechanism budgets.

The enclosure is part of the perception stack

NVIDIA describes post-training Cosmos models for "embodiment-specific tasks, environments, camera or sensor layouts, and policies." Camera layout is therefore part of the model's operating conditions. An industrial design revision that moves a sensor, changes a sightline, or replaces a cover with a material that alters sensing can require recalibration, revalidation, or additional training.

According to Agility Robotics, Digit has moved more than 100,000 totes at GXO's Flowery Branch logistics facility. That deployment began near Atlanta in June 2024, and Agility describes it as the first commercial deployment of a humanoid robot. The capabilities Agility says the deployment helped validate include dynamic balance under varying payloads and vision that can identify and grasp objects repeatedly under different lighting and placement conditions. Lighting, payload, and placement cross the boundaries among environment, mechanism, sensing, and policy. Agility describes Digit's role as connecting "islands of automation," making the robot an interface between existing systems that were not designed to work together.

A robot in routine service is regularly handled: batteries are swapped, grippers are replaced, cameras are cleaned, and covers are removed and reseated. Any of those actions can alter calibration or the sensor conditions on which a policy depends. The technician is therefore a user, and the service procedure is an interface. Good service design makes routine maintenance return the machine to a known state and makes recalibration a repeatable fixture-based procedure rather than an open-ended field-engineering task.

The seam between software cadence and hardware cadence

For example, a team might ship software weekly and release only a few hardware revisions per year. Over a product generation, that could amount to hundreds of model updates while an enclosure changes twice. Those figures are illustrative; cadence varies with product maturity, certification requirements, and deployment model. The general mismatch still calls for stable interfaces between fast-changing software and slower-changing hardware, along with a clear account of what each software version can do on each hardware revision.

1X's NEO illustrates how this cadence intersects with product claims. In October 2025, Humanoids Daily reported that the home humanoid was offered for preorder at $20,000 or $500 per month, with delivery planned for 2026. For tasks NEO cannot perform autonomously, 1X offers Expert Mode, in which a remote human operates the robot. Humanoids Daily, summarizing Marques Brownlee's response to a Wall Street Journal demonstration, reported that all actions in that demonstration were teleoperated. The publication also quoted Figure CEO Brett Adcock criticizing 1X's approach. It quoted 1X CEO Bernt Børnich responding that actions identified as AI in 1X's launch video were autonomous and explaining that the company had chosen not to promise capabilities it could not already demonstrate. In a separate opinion piece, Steve Crowe of The Robot Report argued that teleoperation could be the near-term product and a path toward autonomy, provided 1X is explicit about its limits and privacy implications.

The useful interaction design question is where the autonomy boundary sits and whether people near the robot can see it. Human fallback can be a legitimate system architecture when its role and limits are explicit. Problems arise when a product implies autonomy while relying on an unseen operator, especially when remote operation may expose audio or video from a home. The autonomy boundary is a product surface. It needs visible state, an explicit handoff, and consent that is renewed rather than assumed.

What holds the seams

The design principles that fall out of all this are short enough to list, and they are the working definition of the craft:

1. Name every seam, and write its contract: what is promised across the handoff, at what rate, and what happens when the promise is late.
2. Treat latency as a safety budget allocated across the whole system, not a metric owned by the software team.
3. Co-design the enclosure and the perception stack. A sensor's position changes model inputs, and an industrial design revision that affects sensing requires revalidation.
4. Make the autonomy boundary visible. Design the human fallback as a feature with state and consent, not a secret.
5. Design for the technician. Maintenance must return the machine to a state the model recognizes.

A demo rewards a short period of favorable behavior; deployment exposes a wider range of environments, maintenance states, timing faults, and user expectations. Better models will improve capability, but they do not remove the need to design the interfaces among models, controllers, hardware, operators, and service processes. Treating those seams as explicit engineering objects makes physical AI systems easier to validate, maintain, and trust.