Hybrid physical AI: what inference should stay on the robot

A mobile robot must perceive, plan, and act within physical limits. Keeping all inference on an onboard GPU reduces network dependence, but adds weight, power draw, heat, and a memory ceiling. Sending everything to a server enables larger models, yet makes latency, congestion, and disconnections part of the robot's behavior. Neither location is correct by itself: the unit of decision should be each loop and its maximum tolerable deadline.

Microsoft Research published an offload capability for its Physical AI Toolchain on September 23, 2026 and presented measurements of mobile manipulation across onboard, edge, and cloud GPUs. The announcement is new; the technical report behind the measurements is dated March and is used here as context, not as a result discovered this week. Its figures describe the hardware, models, and network tested by the team, not a guarantee that transfers to every robot.

A useful split keeps deterministic and time-sensitive controls on the robot: stopping, speed and force limits, immediate avoidance, watchdogs, and a minimal safe-motion policy. Semantic perception, global maps, multi-step planning, or policies that need a larger GPU can run on a server at the same site. Cloud remains suitable for less urgent work, elastic capacity, or coordination that can tolerate more variable latency.

The mechanism requires more than moving a container. Every request needs the robot identity, model version, capture time, validity deadline, and an idempotency key. The response must state which observed state it was computed for. If the robot has already moved, a technically correct action may be stale. The client must discard late results and continue with its local policy rather than execute instructions in arrival order.

Build an inventory of decisions, not only models. For each loop, record frequency, worst acceptable latency, input volume, required memory, and the effect of a wrong or missing response. A detector that stops an arm near a person has a different consequence from a model that chooses the next shelf. The first must work locally even when the server, network, or authentication fails; the second can wait, retry, or request intervention.

Treat the classification as a versioned safety contract. An update that makes perception heavier or changes context size must not silently push a local loop onto the network. Require review whenever the deadline, consequence, or minimum capacity changes. The practical result is a placement table with explicit reasons: local required, edge preferred, cloud allowed, or deferred execution.

Measuring only model time is insufficient. The budget includes capture, compression, queuing, upload, deserialization, inference, return, validation, and physical application. Record high percentiles and bursts, not only averages. Microsoft's report warns that added latency can degrade task accuracy and that bandwidth makes naive cloud offload impractical. The network is part of the operating model even though it does not appear in the model weights.

Define a deadline per request and cancel work that can no longer influence the action. Propagate that deadline into the server queue so GPU time is not spent on results the robot will discard. Also separate latency from freshness: a fast response based on an old image can still be dangerous. Observe sensor age at actuation, expiration rate, queue time, and the share of decisions made by fallback.

Fallback must not be an improvised exception. Define what the robot does when a response is missing, three requests expire, or the remote version differs from the expected one. Depending on the case, it may slow down, finish a bounded movement, stop in a safe zone, or request assistance. It should never continue indefinitely with the last plan while the environment changes. Degraded state needs a visible signal and its own telemetry.

Test deliberate failures: total loss, jitter, duplicated packets, a saturated server, expired credentials, and a restart during a task. Verify both transition and recovery, including rejection of responses from the previous session. An approved design is not one that never disconnects in a demo, but one that preserves physical limits when every remote component stops helping.

Imagine a hypothetical robot that moves through aisles, photographs pallets, and flags damage. Wheel control, emergency stopping, immediate person detection, and a short local map stay onboard. An edge server inside the warehouse receives selected images, runs heavier visual detection, and proposes the next inspection point. Cloud consolidates reports and trains new versions outside the movement loop.

If the edge link exceeds its deadline, the robot does not use a late detection to turn. It slows down, finishes the already validated local segment, and stops at the next safe point. It stores thumbnails and events under a retention policy rather than continuous video by default. When the network returns, it reconciles by task identifier and discards old plans. This design preserves utility without confusing remote compute with unlimited physical authority.

Offload expands the data and control surface. Send only the necessary sensors, encrypt in transit, mutually authenticate robot and service, and separate telemetry from sensitive content. Version schemas and models, sign artifacts, and constrain which commands the client accepts. A remote response should describe intent within local limits rather than write directly to actuators. Also review residency, retention, and human access to environmental images.

A shared edge GPU introduces competition among robots. Reserve capacity for deadline-bound tasks, use priority queues, and prevent one noisy robot from blocking the fleet. Measure rejected admissions and design backpressure toward the robot. The official repository uses Kubernetes to distribute containers and includes offload examples, but adopting the framework does not replace capacity planning, network segmentation, or site-specific limits.

Compare at least three configurations on the same task under controlled conditions: all local, hybrid with edge, and degraded mode without a network. Measure complete success, interventions, near collisions, energy per task, observation age at actuation, transferred bytes, GPU occupancy, and operating cost. Segment by lighting, congestion, distance, server load, and link quality. A global average can hide the exact tail where safety fails.

Use published figures as design hypotheses, not as your own business case. The study reports that some small GPUs could not host the full stack and that latency and bandwidth can erase benefits. Your gate must use your actual hardware, network, and task. Promote a version only if it improves the chosen objective without weakening safety limits, and keep independent rollback for the client, model, and remote service.

Hybrid inference is worthwhile when a remote GPU enables a task the robot cannot run within its energy, size, or memory budget, and when that task can tolerate an explicit network contract. It is a poor fit when the team needs perfect connectivity to stop, cannot reject stale results, or still does not know the loop deadline. Separate authority first, then optimize placement.

For a pilot, start with one robot, one repeatable task, and six documented decisions: classification, latency, degradation, data, capacity, and evaluation. If you need to turn that matrix into an architecture and a bounded validation plan, Wasyra can help design the system and its gates without turning a demonstration into a production promise.