Companion analysis · Why agentic robotics is accelerating: NVIDIA's loops and TinyAstral's RSI
A connector misses its socket by a few millimetres. The language instruction was understood; the grasp looked sensible. The remaining problem is how the scene will change as the hand approaches, and whether the next action can correct that change. World-action models try to learn this relationship. Their industrial value will depend on how cheaply it can be adapted and closed into a reliable feedback loop.
HumanWAM and GR00T 2 approach that problem from different experimental priorities. HumanWAM makes data alignment, human pretraining, and visual goals unusually inspectable. GR00T 2 exposes more of the path from mixed pretraining to embodiment adaptation, reinforcement learning, and an agent-assisted runtime. We compare those choices, rather than treating a larger corpus or a higher percentage as an answer by itself. [1] [3] [4]
Evidence is frozen at 12 November 2026: the GR00T 2 whitepaper and Open-HumanWAM v2 are the principal references. “Reported” below means author-reported results, not our reproduction. Engineering recommendations and the final outlook are our interpretation. A missing result means no matched result was established from the reviewed sources, not a zero score. [2] [3]
1. What changes when a policy also predicts the world?
A conventional vision-language-action policy models actions from observations, language, and state. A WAM additionally learns a future-world target. In a generative WAM, actions and future observations can share a conditional distribution. This supplies predictive supervision that encourages consistency between the learned outputs; it is not an explicit physical constraint. Whether that supervision improves control needs a matched ablation. It does not guarantee a causally correct simulator, force estimation, or safe exploration.
VLA: p(A | O, L, s)
WAM: p(O+, A | O, L, s, g?)
L = λvideo Lvideo + λaction LactionHere O is observed history, O+ a future representation, A an action chunk, s proprioception, and g an optional goal. The loss is a schematic description, not a shared implementation: attention masks, noise schedules, and target representations determine whether future prediction actually informs action. Counting both losses is insufficient. A useful ablation removes future-to-action information while keeping training budget fixed.
| Route | Information flow | Engineering consequence |
|---|---|---|
| Joint video–action denoising | Shared generation, e.g. GR00T 2 and OpenWAM-α | Couples physical prediction to control; video tokens also consume runtime |
| Video then action / IDM | Generate future latents, then infer actions; HumanWAM-Vanilla's final recipe | Clear visual bottleneck, but sequential sampling and prediction error propagate |
| Action expert / MoT | Separate action capacity with a chosen cross-stream attention mask | An architectural choice, not a guarantee of joint inference |
| Visual goal conditioning | Desired outcome supplied as clean tokens; HumanWAM-Goal | Clarifies spatial intent; goal creation and reachability become interfaces |
| Human-video ICL | A deployment-time demonstration specifies task order; Zero-WAM | No gradient update, but prompt acquisition and context latency remain |
| World prediction as training supervision | EgoWAM predicts features or motion; world head can be removed at inference | WAM benefits need not require rendering future RGB during execution |
DreamZero established an important generative reference: a 14B video-action model with reported 7Hz control after substantial optimization. That is not a latency guarantee for an arbitrary GPU; its released runtime documents different configurations. EgoWAM asks a complementary question under controlled co-training: are pixels even the best world target? Its DINO and camera-stabilized 3D-flow results motivate testing task-relevant representation, rather than equating photorealism with physical understanding. [11] [12] [18]
Two unrelated projects use the name OpenWAM. We call the 6,400-hour modular pretraining model OpenWAM-α. Separately, Stanford's composable framework compares joint, video-first, action-first, and decoupled programs on a shared setup. Its LIBERO results are useful counterevidence to an absolute claim that joint denoising always wins. The model identity and protocol must travel with every number. [15] [17] [25]
2. GR00T 2 versus HumanWAM: compare three configurations
“HumanWAM” is not a single architecture. Vanilla uses a Wan2.2-TI2V-5B video expert plus an action expert. The final Vanilla policy uses inverse-dynamics attention and video-then-action inference; the isolated-attention scaling experiments can skip future video at execution. Goal switches to a Cosmos3-Nano-16B backbone and a shared generator. Mixing these configurations would attribute changes in architecture and data to visual goals alone. [3] [26] [27]
| Dimension | GR00T 2 | HumanWAM-Vanilla | HumanWAM-Goal |
|---|---|---|---|
| Backbone scale | Cosmos-Predict2.5-2B shared DiT | Wan2.2-TI2V-5B + action expert; total not simply 5B | Cosmos3-Nano-16B backbone; frozen Reasoner plus Generator |
| Training target | Joint video/action flow matching; mask invalid actions | Video/action denoising with attention ablations | Goal-conditioned joint video/action generation |
| Action interface | Embodiment-specific MLP encoders/decoders; robot-specific control spaces | 16-D bimanual EEF: position, quaternion, openness | 49-D hybrid layout; unused dimensions masked; head slots are metadata |
| Human learning | Human + UMI + robot mixture in staged training | Pure human pretraining, then EgoPro and robot adaptation | Hybrid human–robot co-training; visual goals |
| At inference | Shared future/action sampling; per-robot normalization | Final recipe samples video before actions | Goal/history tokens; joint sampling; optional goal omission |
| Demonstrated bodies | YAM, Franka, Dexter/Sharpa, other hands, G1 | Bimanual simulation and task-adapted hardware | Aloha-AgileX simulation; Astribot S1 hardware |
| Interpretation | Broad execution stack with several adaptation paths | Inspectable human-data and transfer recipe | Outcome interface and hybrid-data scaling experiment |
Parameter labels require a denominator. GR00T 2's quoted 2B describes its shared transformer, not the entire sensor-to-actuator system. The whitepaper's tested Turbo configurations also use 2B; their efficiency gain includes video compression and runtime changes. HumanWAM's 5B and 16B labels describe different backbones, not matched counts of trainable policy parameters. None of these labels normalizes camera tokens, reasoning encoders, denoising steps, or hardware. [2] [3]
Our technical reading is that the choice of action interface matters more than the parameter headline. A common EEF space makes demonstrations comparable, but hides joint limits, finger contacts, and compliance. A body-specific decoder preserves those details but moves work into adaptation. Neither approach removes embodiment: it chooses where to pay for it.
GR00T 2
HumanWAM-Vanilla / final
HumanWAM-Goal
Conceptual reconstruction from the reports, not copied network diagrams; attention and execution differ by configuration.
3. A data hierarchy, not a competition in recorded hours
| Source | Useful supervision | Cost and transfer burden |
|---|---|---|
| Human ego video | Scene change, intent, hand motion; reconstructed poses vary in quality | Portable collection; curation, tracking, motion feasibility and occlusion remain expensive |
| UMI / handheld grippers | Wrist-relative trajectory, measured gripper aperture, robot-like contacts | Less robot occupancy; calibration, camera geometry and latency matching matter |
| DexUMI / wearable hand | Finger kinematics and hand–object interaction, depending on the device | Richer contacts, more wearable hardware, retargeting and compliance work |
| Robot teleoperation | Actual embodiment, actuator response, deployment camera and state | Robot/operator time and resets; captures failures specific to the machine |
UMI's original contribution includes relative trajectories and latency matching, not simply attaching a camera to a gripper. DexUMI adds hardware and software measures to reduce the hand gap. Therefore, Human Ego, UMI, and DexUMI are different observation–action contracts. A shared file format does not make their actions interchangeable. [8] [9] [10]
| Dataset / role | Reported scale | What must not be conflated |
|---|---|---|
| GR00T 2 Human-1M | About 1M hours collected; 402,730 hours in the model inventory | Raw inventory, filtered training mixture and 100K release scope differ |
| GR00T 2 UMI-10K | 10,184 hours; 942K accepted episodes; 2,897 tasks | Robot-like grasp supervision, not ordinary bare-hand video |
| GR00T 2 robot teleoperation | 23,500 hours across multiple embodiments | Collection proportions are not stage-wise sampling weights |
| Open-HumanWAM-50K | 50,000 curated hours; 35.9M selected clips | Five-source corpus; clips can overlap within physical episodes |
| HumanWAM EgoPro bridge | Approximately 1K hours in the source; 724 retained in the 50K corpus | Synchronized head + wrists; do not add it again as disjoint new data |
| HumanWAM-Goal hybrid experiment | 14,618 robot hours + 45,938 human hours in the largest reported mixture | Not the Vanilla pure-human pretraining recipe |
The Human-1M release card preserves timestamps, rig identity, calibration, and source metadata. Open-HumanWAM preserves clip manifests and provenance; its 50K hours are a selected corpus, not 35.9 million independent physical trials. These details make a scaling result auditable. For procurement, cost per accepted, usable interaction is more meaningful than camera-on hours. Neither project provides a complete comparable dollar-per-hour acquisition bill. [5] [7]
4. Head and wrist views: alignment before camera count
The head camera preserves workspace context, object relations, and task intent. A wrist camera makes the grasp site larger in the image and can see contact hidden by the other arm. It also moves with the effector, narrows the field of view, and can lose the destination during transport. Head + wrist is therefore a coverage choice with calibration and token costs, not automatically a stronger model.
HumanWAM supplies a particularly useful negative result. In its isolated-attention ablation, adding wrists only during robot post-training damages scene-OOD and random performance. In the IDM setting, EgoPro mid-training partly repairs the transition; observing all views while predicting only the head view is better than predicting every view. These are matched ablations inside that study, not evidence that all multi-view policies fail. [3]
| HumanWAM ablation | Clean % | Scene-OOD % | Random % |
|---|---|---|---|
| Isolated / head only | 75.68 | 46.08 | 22.48 |
| Isolated / add wrist conditioning | 74.60 | 34.08 | 12.92 |
| IDM / head only | 88.48 | 62.00 | 28.56 |
| IDM / head + wrist conditioning | 90.56 | 59.52 | 29.08 |
| Same + EgoPro mid-training | 90.28 | 61.20 | 30.68 |
| IDM / predict all views + EgoPro | 88.72 | 58.80 | 27.36 |
Table: HumanWAM v2, Table 5. Compare view changes within an attention/action configuration; crossing the isolated and IDM blocks also changes action parameterization. These are recipe ablations, not the final 50K benchmark scores. [3]
Synchronization has a physical unit. As an illustrative calculation, 20ms of image–action skew at 0.25m/s means 5mm of displacement, already larger than many insertion tolerances. A collection pipeline should record exposure and state timestamps, estimate offsets and drift, preserve camera-to-effector transforms, and verify alignment during motion. Matching nominal frame rates is insufficient; rolling shutter and buffering can still shift the contact event.
Generalist's data strategy is relevant because GEN-1.5 physical prompts normally contain sensorimotor examples from handheld grippers or robot rollouts. We use “wrist-first” as an engineering interpretation: prioritize a consistent near-hand observation–action interface. The reviewed official posts do not establish an exclusively wrist-only camera stack, an all-else-equal comparison against head + wrist, or a disclosed WAM video-generation objective. GEN-1.5 is an important adaptation reference, not a confirmed architectural twin of GR00T 2. [19] [20]
5. Visual goals, video prompts, and gradient updates are different adaptation tools
A goal image specifies a state. A video specifies transitions and ordering. A sensorimotor prompt additionally supplies actions. This hierarchy matters when the desired final picture is compatible with several incompatible procedures: a closed lid does not say whether to screw, press, or latch it. HumanWAM-Goal's outcome interface can help an agent communicate an intermediate target, but a selected goal-following clip does not prove procedure learning from arbitrary human video. [4]
Zero-WAM explicitly trains use of human-video context. HumanGen generates semantically paired human videos from robot trajectories, and the in-context future-chunk objective discourages shortcuts. The reported 46.95% is on seven held-out RoboTwin tasks, not the 50-task post-training suite. This reduces one task-specification gap while introducing another audit problem: generated human videos must preserve the task and its physical ordering. [13] [14]
GEN-1.5 reports 59% ±10% one-shot ICL and 83% ±9% after ten gradient steps using five minutes per task, across ten short-horizon tasks; the reported uncertainties are standard deviations, not confidence intervals. Its 100Hz action trajectories are not a measured 10ms full-model inference latency. “No updates,” “a few updates,” and “zero target-robot task demonstrations” answer different questions, and should be three separate columns in any adaptation claim. [19]
6. Benchmarks: keep the embodiment and denominator attached
RoboTwin 2.0 evaluates bimanual Aloha-AgileX tabletop manipulation; Clean2Random introduces visual/environment variation after clean-demonstration training. RoboDojo probes bimanual generalization, precision, long horizon, memory, and open instructions; progress and complete success are separate metrics. LIBERO's four suites are not a replacement for either. RoboCasa365 has 365 tasks in its environment collection, while the cited leaderboard experiment evaluates a 50-task selection, not 365 task successes averaged together. [21] [22] [23] [24] [2]
| RoboTwin: clean-only post-training | Clean2Clean % | Clean2Random % | Interpretation |
|---|---|---|---|
| HumanWAM-Vanilla | 92.08 | 68.88 | Pure-human pretraining, then robot SFT |
| HumanWAM-Goal | 92.70 | 84.60 | Hybrid pretraining + goals + robot SFT; several factors change |
| π0.5 / reported baseline | 70.70 | 46.00 | Transcribed by the HumanWAM report; not our retraining |
| GR00T 2 | No matched row | No matched row | Do not substitute GR00T-N1.7's score |
The HumanWAM results use 50 clean demonstrations per task and 100 evaluation episodes per task for Vanilla. Goal's zero-shot RoboTwin mean is instead 13.94%; its post-trained 84.60% Random result is not zero-shot. The paper's Vanilla RoboDojo mean, 14.17% success / 19.00 progress, equally weights six groups by separating standard and random generalization. GR00T 2's official three-layout-bank RoboDojo result is 35.52% success / 41.73 progress with a five-dimension aggregate. Those aggregate means are not a controlled head-to-head result. [2] [3] [27]
| GR00T 2 configuration | RoboCasa365 SR % | RoboDojo bank 0 SR % | System being evaluated |
|---|---|---|---|
| SFT | 67.2 | 35.2 | Policy |
| SFT + RL | 72.4 | 40.8 | Post-trained policy |
| SFT + Agent | 81.8 | 72.3 | Programs, executor selection, online recovery |
| SFT + RL + Agent | 88.0 | 75.8 | Combined system |
Table: GR00T 2 whitepaper Table 19, a 6 November leaderboard snapshot. The RoboDojo rows here use bank 0 only. The agent chooses its default executor using development seeds and is tested on disjoint seeds; it is not merely the same policy with a longer language prompt. The large memory and open-task gains suggest an interface/recovery bottleneck, while precision gains from RL point to an execution bottleneck. That attribution is our reading of the within-system changes, not a proof that every memory task needs an agent. [2]
For LIBERO, OpenWAM-α reports 99.3% average; Stanford's different OpenWAM reports 98.6% for VTA and 97.6% for Joint, with three seeds and 50 episodes per task. High scores leave limited room to distinguish methods. The α project's separate RoboCasa-GR1 result must not be relabeled RoboCasa365. Before chasing a 0.5-point lead, compare reset rules, checkpoint selection, data access, and confidence intervals. [16] [17]
| Physical evaluation | Reported outcome | Boundary |
|---|---|---|
| GR00T 2 / Dexter / policy / zero-shot | 49.8% SR on 71 tasks, 3 trials/task; π0.5 26.3% in the same sessions | Small per-task sample; do not confuse with Dexter-50 |
| GR00T 2 / Dexter-50 / LoRA | 40% zero-shot → 71% one episode → 95% five episodes | Tasks selected with few-shot learnability; target-task absence is body-specific |
| HumanWAM-Goal / Astribot S1 / zero-shot | Largest human mixture: 42% easy, 19% hard, 10 tasks/group and 10 trials/task | Goal-conditioned 20-task suite; not the Dexter task distribution |
| GEN-1.5 / reported short tasks | 59% ICL; 83% after adaptation | Different demonstrations, bodies and trial protocol; not a WAM ranking row |
7. Post-training: where GR00T 2 closes the contact gap
Robot SFT grounds a broad model in the target machine's camera, state, and action distribution. LoRA lowers the cost of updating the transformer, not the cost of collecting trustworthy examples. For Dexter-50, the report uses rank-32 adapters, one or five task episodes, and roughly 30 minutes on one RTX 5090 per task. Frozen encoders and heads still require compatible observations and normalizations. This is efficient adaptation under a defined interface, not plug-and-play control of an unknown body. [2]
Can UMI replace a target robot's same-task demonstrations? In the reported UMI→YAM experiments, yes, conditionally: UMI supplies task behavior while YAM contributes task-agnostic data. Shared action heads, grasp-site transforms, aperture normalization, and YAM-aligned wrist rendering bridge the gaps. Toy-truck assembly reaches 60% with aligned views but 0% without warping in the reported ablation. The result removes one demonstration source; it does not remove the target body's calibration, play data, controller, or validation. [1] [2] [10]
The three RL options spend interaction differently. Group-relative RL changes the base WAM, using stochastic action-denoising transitions and reward comparisons; its context-history baseline retains information across rounds. Residual RL freezes the base and learns bounded corrections, making rollback and local specialization easier. Self-imitation collects successful RL-discovered episodes and starts again from the SFT initialization with supervised video/action losses. It inherits exploration cost even when the consolidation phase needs no new rollouts. [2]
| YAM precision task / SR % | SFT | Residual RL | GRPO | Self-imitation |
|---|---|---|---|---|
| Fixed-pipe fitting | 66 | 76 | 86 | 88 |
| Finger placement | 58 | 84 | 72 | 98 |
| Ethernet insertion | 76 | 82 | 92 | 94 |
| Charger insertion | 40 | 70 | 94 | 96 |
Table: whitepaper Appendix F.4, 50 trials per task/method, matching resets and time limits, no human evaluation-time intervention. The authors find GRPO improvements statistically conclusive at 5% for pipe and charger tasks, not for the other two at this sample size. Residual training uses human-assisted experience; self-imitation reuses successful GRPO rollouts. Evaluation autonomy must not erase training supervision. [2]
Isaac Lab contributes another data path, not a substitute name for all RL. In the human→sim→real example, a state-based expert learns reconstructed object motion with randomized physics; perturbed rollouts include corrections, are rendered, then supervise GR00T 2. The report uses 1.5K simulated episodes per task. DreamDojo neural-simulation RL is a separate path and needs recorded task rollouts to build its environment. A generated future inside a policy is not automatically an environment calibrated well enough for policy optimization. [1] [2]
Foundation
Adaptation branches
Execution and evidence
Schematic deployment proposal. The branches are alternatives or staged experiments, not a claim that every checkpoint underwent every method.
8. Deployment is a timing and reproducibility problem
| Runtime measurement | Reported value | Timing boundary |
|---|---|---|
| GR00T 2 / Dexter / RTX 5090 | 152.78ms median / 48-action horizon | Optimized timing excludes prefetched VAE encoding |
| GR00T 2 Turbo / G1 / RTX 5090 | 38.40ms median | 1 video / 2 action steps; VAE included |
| GR00T 2 Turbo / G1 / Jetson Thor 120W | 123.97ms median | 1.6s plan, about 1Hz replanning; SONIC at 50Hz |
| HumanWAM-Goal / PhyAI | 2.2× versus its PyTorch + CUDA Graphs baseline | No matched absolute latency/hardware table established here |
| DreamZero / optimized paper configuration | Reported 7Hz | Different model, devices and caching; release timings vary |
Latency is not the inverse of action sampling rate. A 1.6-second trajectory can contain many high-rate setpoints yet be refreshed only once per second. Camera exposure, encoding, network transport, queueing, denoising, and controller tracking all contribute to response. Report p50 and p95 sensor-to-command age under load, missed deadlines, and recovery after stale chunks. Asynchronous serving needs prefix alignment or another explicit continuity contract.
Reproduction should pin the checkpoint, dataset manifest, per-body action convention, calibration, normalization statistics, camera ordering, denoising schedule, simulator version, and evaluation seeds. Publishing a repository is only the beginning. HumanWAM's source datasets retain their original terms; a wrapper's Apache license does not relicense them. GR00T's published human-data subset is not the complete training mixture. Full pretraining resource disclosure is also uneven: GR00T reports 1,024 GB300 GPUs, whereas no like-for-like full training bill was established for HumanWAM from the reviewed release. [2] [5] [6] [7] [25]
Our proposed deployment sequence is deliberately testable: calibrate and replay recorded motions; establish a robot-SFT baseline; test task-agnostic and same-task UMI transfer separately; add RL only to a measured failure mode; then add agent recovery with an explicit retry and time budget. Freeze a holdout set before each step. Promote a change only if complete-task success improves without an unacceptable increase in cycle time, intervention, damage, or compute cost.
9. What deserves attention over the next one to two years?
Our first priority is accepted-interaction scaling. More human hours are useful when they add new contacts, objects, procedures, and recoveries after curation. Hold robot data and compute fixed, then test unseen scenes and tasks. Report the marginal acquisition and processing cost. Otherwise a scaling curve can conceal better sampling or a larger adaptation budget.
Second, evaluate unified actions as a hierarchy. EEF trajectories are a useful shared geometric interface; dexterous contacts and whole-body balance may need body-specific residuals, force targets, or motion tokens. A useful cross-body test holds the task fixed, counts every target-body calibration and play episode, and measures how much of the prior survives without re-learning contact from scratch.
Third, compare wrist-only, head-only, and combined rigs under equal accepted task hours and model-token budgets. Measure occlusion failure, destination loss, recalibration, and transfer to a different hand. We expect wrist-first interfaces to be attractive for bounded contact work, and wider views to remain useful for task composition and navigation. This is a falsifiable design hypothesis, not a winner established by the existing cross-paper scores.
Fourth, reduce future prediction to what control needs. Sparse visual futures, latent dynamics, asynchronous updates, and task-dependent denoising could improve reaction cost. The acceptance test is physical success at matched latency, including precision and perturbation recovery, with an action-only ablation. A fast model that loses the failure signal is not a successful compression.
Finally, WAM + RL + Agent should be evaluated as an accountable system. The WAM proposes a physically grounded action chunk; RL improves a repeatable contact error; the agent detects a broken precondition, changes a subgoal, or retrieves a recovery. Each layer needs an observable contract and an ablation. Track complete-task reliability over repeated shifts, not only best-run success after unconstrained retries.
Our judgment: a promising training interface, an unfinished deployment contract
The strongest case for WAM is that broad human experience can supervise more than object semantics: it can constrain how intended motion changes a scene. HumanWAM makes that case legible through alignment, controlled views, and goals. GR00T 2 makes it operationally concrete through cross-adaptation, contact post-training, and runtime layering. Neither establishes that generated video is sufficient for reliable manipulation, or that an arbitrary new robot can skip grounding.
For a research group, follow HumanWAM when the question is what data or conditioning buys transfer; use matched ablations rather than its headline score. Follow GR00T 2 when designing the adaptation and evaluation stack, while accounting for resources and release scope. For deployment, start with one costly task, compare against a strong VLA under the same data and latency budget, and require evidence that predictive modeling reduces total cost at the required reliability. That is a more durable milestone than winning a mixed leaderboard.
Follow the data contract down to the contact loop
The UMI device study tests the collection side of the argument: hours are useful only when timestamps, coordinates, gripper state, and accepted-trajectory yield support the chosen action interface. The Sharpa study tests its execution side: video pretraining does not by itself provide tactile feedback. These articles complement this comparison without adding independent evidence to its benchmark tables. Our proposed next step is a matched data-source and sensing ablation on one robot, with adaptation and intervention costs included.
Primary sources and reproducibility
- GR00T 2: official project
- GR00T 2 whitepaper, §§2–4 and appendices A, C, E–K
- Open-HumanWAM report v2
- Open-HumanWAM: official project
- Open-HumanWAM-50K: corpus, subsets, and source terms
- HumanWAM released checkpoints
- Human-1M-100K: official dataset card
- UMI: original interface and latency matching
- DexUMI: original hand interface
- GEAR UMI hardware documentation
- DreamZero: paper
- DreamZero: release and runtime
- Zero-WAM: paper v2
- Zero-WAM: data pipeline and evaluation
- OpenWAM-α: modular pretraining project
- OpenWAM-α: full benchmark tables
- OpenWAM: Stanford composable framework
- EgoWAM: controlled world-target study
- GEN-1.5: official experiments
- Generalist GEN-0: data-engine background
- RoboTwin 2.0 official leaderboard
- RoboDojo: benchmark paper
- RoboCasa365: benchmark and versions
- LIBERO benchmark
- OpenWAM-α: implementation and deployment
- HumanWAM-Vanilla model card
- HumanWAM-Goal model card