A person can complete a delicate task with a handheld gripper while the resulting recording remains poor robot supervision. The camera may arrive late, the two hands may use different clocks, or a comfortable human wrist path may exceed the arm’s reach. Adding a force sensor helps only if its measurements stay aligned with the action and the robot can use them at execution time. Useful UMI data is an engineering outcome.

The thirteen devices invited to the CoRL 2026 UMI Arena make this change visible. Some improve pose labels and processing, some preserve contact, and others extend the action space to fingers or whole-body motion. Specialist deserves particular attention because its hardware, geometry pipeline, and deployment interfaces expose many of these tradeoffs in inspectable code. It offers a concrete starting point for asking what a data collector must preserve for a robot to learn. [1]

Evidence checked on 16 November 2026. D01–D13 are workshop device identifiers, not a ranking or thirteen matched benchmark results. We distinguish official listings, author-reported experiments, and 1N Robotics recommendations. Specialist’s paper and some learning/control repositories remain marked “coming soon” at this check; public hardware and reconstruction code do not establish a fully reproducible end-to-end release.

1. UMI’s lasting idea is an observation–action interface

The original Universal Manipulation Interface (UMI) combines handheld grippers, wrist cameras, relative trajectory actions, and inference-time latency matching. Demonstrators manipulate real objects without having a robot beside them. A compatible robot then receives observations and motion targets in a comparable form. This reduces collection dependence on robot availability, while retaining constraints on gripper geometry, feasible motion, and execution timing. [2]

Current devices improve different parts of that interface. Localization improves action labels; encoders measure jaw configuration; tactile and audio channels expose contact; morphology determines which demonstrations are possible. A data pipeline determines whether those signals become accepted episodes or expensive files awaiting repair. Counting sensors or recorded hours alone misses these dependencies.

2. Thirteen devices, organized by the bottleneck they address

DeviceMain directionEvidence and selection boundary
D01 SpecialistMatched fingers/camera, dual-fisheye reconstruction, printed anchorsPublic hardware and reconstruction; incomplete learning/control release
D02 UMI-LiveBimanual LiDAR–IMU capture and asynchronous processingDetailed full workflow; one autonomous evaluation task
D03 FLEXIndependent jaw actuation and integrated multimodal sensingOfficial device description; no matched quantitative comparison here
D04 KoalaCo-design capture hardware and a more capable robot gripperA morphology choice, not a drop-in parallel-jaw data upgrade
D05 EucalyptusDedicated tracking, jaw encoder, two finger load cellsDirect force measurements; tethered capture and limited force ablation
D06 PolyUMIWrist vision, optical touch, and contact audioContact-focused sensor ablations and a shared sensing finger
D07 GDataWearable/handheld robot-free capture; head–hand localizationWorkshop descriptions establish direction, not measured accuracy
D08 YUBI GoQuest capture software for finger-driven YUBI grippersSeparate the Go app from the wider YUBI corpus and models
D09 HoMMIHead–hand observations and whole-body mobile executionPolicy and controller design beyond a handheld device
D10 YAM-UMIHand-worn, fiducial-tracked YAM-compatible interfacePublic designs; inspect marker visibility and pose setup
D11 PopeyeInfer hand pose/contact force from forearm surface imagingAn estimation route; do not equate it with direct load-cell truth
D12 Aero-UMIMatched five-finger exoskeleton and tactile/force captureVendor specifications need independent task-level validation
D13 TRIMIDirect demonstration, haptic teleoperation, robot executionKeep the three modes and their data provenance distinct

The official roster is the source for the device identities and broad descriptions. For FLEX, GData, Popeye, and TRIMI, this review does not establish independently reproducible cost, latency, or complete-task success figures. We retain their research directions without inventing missing specifications. Detailed evidence for the other entries follows below and in the linked repositories. [1] [17]

3. Specialist: separate the camera used to act from the camera used to label

Specialist mounts an Insta360 X3 on each gripper. Its two back-to-back fisheye lenses have different jobs: one observes the fingertips and workspace for policy learning, while the other sees room features and stationary AprilTag anchors for reconstruction. Tags on the gripper body and fingers help recover finger configuration; cross-gripper sightings connect the hands. The actuated robot version preserves the handheld version’s fingers and camera placement. [3] [4]

The useful design principle is to give the labeling pipeline more evidence without forcing every labeling sensor into the deployed policy. During an insertion inside a cabinet, the task-facing image may contain almost no trackable room structure. The auxiliary lens can still see the room. This reduces competition between looking at the contact and looking at landmarks, although it still depends on texture and tag visibility. [3]

Specialist: capture, reconstruction, and execution have different inputs

Capture

Two X3 lenses + IMU + audio on each gripper
Fixed room anchors; tags on the gripper and fingers
Keep both hands’ raw recordings and calibration

Offline labels

Audio clock alignment → visual–inertial/tag factor graph
Metric poses, finger state, timestamps → training rows
Quality checks and robot feasibility screening

Robot execution

Matched camera placement and compliant fingertips
Task-facing image → learned policy → feasible motion
Measure camera age and actual tracking error

The first two lanes summarize the public pipeline; quality gates and feasibility screening are our recommended acceptance steps

This is a controlled correspondence between capture and execution, not proof of universal embodiment transfer. Changing fingertip stiffness, jaw travel, camera extrinsics, or controller bandwidth changes what an apparently identical pose means physically. Specialist’s co-design narrows these differences; transferring to an arbitrary existing gripper reopens them.

4. What Specialist actually reconstructs

The public reconstruction code converts the per-lens recordings into a shared session, aligns cameras using audio, and jointly optimizes visual odometry, IMU preintegration, and tag-corner reprojection in GTSAM. An anchor fixes the world frame and metric scale. Multi-gripper tag observations link trajectories. The base recording, estimates, visualization, and training-state layers are stored separately, so geometry can be regenerated without treating every derived result as raw truth. [4]

The setup guide specifies a 100.0 mm inner anchor square printed at actual size. A printer’s scale error propagates directly into distances. Anchors must stay rigid and visible; shadows can prevent tag detection. Thus “no external tracking rig” still has an environmental preparation cost. This is a reasonable bargain for many benches, and a constraint for cluttered spaces where the operator repeatedly blocks the rear view. [5]

The dataset writer interpolates poses, velocities, and finger state onto 100 Hz rows while retaining actual image timestamps. Those rows are not one hundred independent camera measurements per second, and they do not establish a 100 Hz visual policy. The checked documentation and repository README also show different image-reduction defaults, 240 × 240 and 360 × 360. Pin a commit and record the chosen scale rather than copying a default across versions. [4] [6]

Sound serves as a clock cue here. It should not be counted as a contact-audio policy channel merely because the files contain audio. PolyUMI’s contact microphone has a different role: it supplies task observations about interaction. Likewise, a tag-derived finger angle is a configuration label, not a direct force measurement.

5. Specialist’s evidence is promising, with specific boundaries

Author-reported itemValue / settingHow to interpret it
Tracking3.0 mm median; 6.6 mm p90; 11 sessions, 19,716 frames against motion captureA tracking evaluation, not autonomous manipulation success
Collection timeLEGO assembly: 218.2 → 136.5 s, eight operators, versus UMITask-specific ergonomics evidence; not end-to-end cost reduction
Handheld unitListed $320, 315 g; about five print hours and six assembly minutesAn author-listed unit estimate, not a complete lab budget
Robot gripperListed $380, 480 g; about 7.5 print hours and 20 assembly minutesAdd arm, processing machine, setup, and validation labor
Camera benchmark720 × 720 at 30 fps: 153.7 ms median, 172.7 ms p95 glass-to-visible, n = 1,000Bench flash/display method includes measurement bias; not full sensor-to-actuator latency

Tracking, user-study, and build figures are published on the project site; the camera figures come from the driver’s documented benchmark. The original UMI’s separately reported mean error is not a matched baseline for Specialist’s median. Camera timing includes display and flash-period effects. Even with those qualifications, a roughly 150 ms image path is consequential for reactive contact: a faster policy cannot recover information that has not arrived. [3] [7]

Its force story is also precise. The device records finger position; compliant fingertips help reproduce grip behavior, but there is no measured force channel in this version. A force-replay experiment cannot be read as force-supervised learning or reliable regulation of insertion loads. Add direct sensing when force is part of the task definition, and test whether the deployed controller can respond to it. [3]

The release is useful but uneven. Hardware descriptions, reconstruction, simulation, gripper/servo drivers, and the camera reader are public. The organization’s index still marks the LeRobot plugin and behavior-cloning trainer as coming soon, and the referenced arm-control stack is absent from its public repository list. Autonomous videos show feasibility, but a complete-task evaluation table and a fully public training-to-robot recipe are still needed before claiming turnkey reproducibility. [8]

6. UMI-Live: a strong workflow baseline, with a weight and validation bill

UMI-Live integrates a RoboSense Airy LiDAR–IMU, global-shutter wrist camera, Raspberry Pi 5, and battery in each unit. Its evaluated pair costs $1,930; each device weighs 1.42 kg with battery and runs about 1.5 hours per charge. This is accessible compared with keeping two robot arms at every collection station, but it is a different ergonomic proposition from Specialist’s 315 g handheld unit. Cost comparisons must keep per-pair and per-unit prices separate. [9]

Its engineering attraction is the background pipeline. Recording, uploads, LiDAR–inertial reconstruction, gripper estimation, checks, and LeRobot assembly are coordinated while collection continues. The code supports alignment of the two trajectories into a common metric frame; keeping two independent episode frames is a different configuration and cannot silently provide inter-hand geometry. No environment calibration does not eliminate one-time camera and jaw calibration. [10]

The released dataset contains 3,255 bimanual episodes, about 39.7 hours at 30 Hz, across orange placement, water pouring, and towel folding. Its dataset card counts paired time once and says the merged release retains all source episodes without filtering or deduplication. Treat dataset publication and physical quality acceptance as separate checks. [11]

Dual-YAM orange taskUMI / teleoperation mixtureFull success
π0.5, fine-tuned95% / 5%9/10
π0.5, fine-tuned90% / 10%10/10
MolmoAct2, fine-tuned75% / 25%8/10

Success requires all six oranges in the bowl. Each condition uses 12,000 updates, global batch size 32, ten physical trials, and one training seed. Robot teleoperation comes from the evaluation robot and scene. With no UMI-only and teleoperation-only comparison, these results do not show that UMI fully substitutes for target-robot demonstrations. The project’s match-lighting teleoperation setup also uses Vive tracking rather than the LiDAR capture path. Our recommendation is to reproduce this workflow as a candidate baseline, not declare it the most mature of all thirteen devices. [9] [12]

7. YUBI: study the data operation, and distinguish the Go application

The June YUBI paper reports 8,434 hours, 1.20 million episodes, and 119 tasks. The November AIRoA-0 report uses 16,151 hours and 10.65 million subtask segments with task/subtask language. These are successive corpus descriptions, not quantities to add. They support studying YUBI as a large-scale collection ecosystem; they do not establish that YUBI Go itself collected the whole corpus. [13] [14]

Go is specifically an open-source Rust application on Meta Quest 3/3S. Two YUBI grippers contribute wrist video, angle encoders, and Quest-controller tracking through a hub; a head camera is optional. Episodes contain MCAP signals and metadata, with LeRobot export and optional backend/upload integration. The repository calls it research software and documents unauthenticated local preview services. A production data platform needs an access-controlled network and an explicit operator/site data policy in addition to devices. [15]

AIRoA-0 is an important downstream result, but its headline needs care. The paper’s ten-task average is 78 versus 27 for the strongest compared model; each task uses fifteen trials and grants partial credit. We therefore describe it as a partial-credit task score, not 78% strictly complete autonomous episodes. It also includes task-specific handheld post-training. No newly collected robot data does not mean no target-task data, and the comparison evaluates the whole training/inference system rather than the gripper alone. [14]

For a team building a data platform, the lesson is to measure accepted episodes per collector-day, task coverage, and the marginal gain from adding a new site or operator. Hardware count matters only when storage, processing, language annotation, and evaluation keep up. Record failed and corrective attempts separately; a corpus of smooth human demonstrations does not automatically teach a robot how to recover from its own errors.

8. Eucalyptus and PolyUMI answer different contact questions

Eucalyptus replaces image-only jaw estimation with a magnetic encoder, combines two three-axis finger force sensors with a dedicated Xvisio DS80 pose tracker, and keeps wide-angle wrist RGB. Its workshop paper reports more than 18,000 demonstrations collected by five people using four devices over summer 2026. Capture is tethered to a host. The dedicated tracker remains visual–inertial SLAM, not an independent motion-capture ground truth, and the paper explicitly leaves the downstream benefit of force observations for systematic evaluation. [16]

PolyUMI instead preserves a contact patch and its vibrations. The shared sensing finger records optical tactile images and contact audio alongside wrist video and pose, then moves to the robot. VisTA fuses modality tokens and predicts action chunks. Its screwdriver slip-control ablation reports 8/10 with vision, touch, and audio versus 2/10 with vision alone; success includes catching the tool within 10° of vertical without dropping it. This is a small controlled physical contact test, not a multi-task reliability estimate. [18]

Choose the measurement for the failure. Net finger force helps analyze load and force limits; a tactile image shows local contact geometry; audio detects brief vibrations that a slow image stream may miss. They are complementary, but each adds synchronization, maintenance, and model-support costs. Training on a modality absent from the robot needs an explicit distillation or missing-sensor strategy, not a hope that extra data will transfer.

9. Beyond parallel jaws: morphology changes the data contract

Koala co-designs the human capture device and robot gripper around grasp capabilities beyond a parallel jaw, including independently controlled fingers opposed by a pivoting dual thumb. That changes which tool actions can be demonstrated and executed. It also commits the team to a different action space and hardware stack. HoMMI expands a different axis: head sensing, a cross-embodiment hand–eye policy, and a constraint-aware whole-body controller for mobile manipulation. More cameras alone do not provide that coordination. [19] [20]

Aero-UMI is a candidate when five-finger contact and joint-level demonstrations are essential. Chestnut describes a kinematically matched exoskeleton and Aero Hand with direct joint encoders and distributed force capture. Its page lists fifteen encoded joints within an eighteen-joint structure and claims sub-millisecond cross-stream alignment. These are vendor specifications; joint count, independent control degrees of freedom, encoder repeatability, and task success are different quantities. We did not find a matched autonomous benchmark sufficient to rank it above other dexterous interfaces. [21]

YAM-UMI provides a more focused alternative: a hand-worn interface matched to a YAM linear gripper, with fiducial aperture sensing and configurable pose tracking. FLEX explores independent jaw control; Popeye estimates hand state and force from a different bodily signal; TRIMI connects direct demonstration with haptic teleoperation and robot execution. These paths should be selected by the required action and contact representation, rather than added to one sensor-count leaderboard. [1] [17]

10. Fisheye-VLA and CamVLA improve different parts of observation geometry

Fisheye-VLA uses one fixed 220° fisheye to generate a global view and two end-effector-centered perspective crops, with ray encoding to preserve spatial meaning. It allocates pixels already captured by the sensor; a virtual crop does not recover detail hidden by an occluder or absent from the source image. The reported interface uses calibrated geometry. This is different from Specialist’s moving wrist-mounted dual-lens cameras. [22]

CamVLA separates camera-frame action prediction from a geometric head estimating the camera-to-base relation, then transforms the action deterministically. Deployment avoids supplying external camera calibration, while training still provides geometric supervision and viewpoint diversity. The method addresses pose robustness; it does not create wider field of view, observe hidden contact, or remove mechanical calibration elsewhere in the robot. [23]

ExperimentAuthor-reported comparisonConditions and limit
Fisheye-VLAR2: 84% vs 58%; R3: 82% vs 14% for front + two wrists50 trials/configuration/region; expanded tabletop, some targets outside baseline coverage
CamVLA / π0At 15°: 29.3% vs 16.0%Five physical tasks, three test cameras, twenty trials/task/camera/offset
CamVLA / GR00TAt 15°: 33.0% vs 14.7%, GR00T N1.7Same physical protocol; success is still low at this shift

Sources are the papers’ closed-loop tables. Fisheye-VLA’s R1 result is 72%, below the front-plus-wrists baseline’s 92%; its advantage depends on the workspace. Each sensing configuration uses the same 300 teleoperated demonstrations per task. CamVLA trains with five camera perspectives and 100 demonstrations per task per training viewpoint; CamVLA and its baselines still perform poorly at larger real camera shifts. The often-quoted 65.3 → 6.3 collapse is CamVLA’s RLBench simulation example, not its physical experiment. [22] [23]

Combining UMI-Live collection, Fisheye-VLA observations, and CamVLA-style geometry is a research proposal, not an available universal system. Wrist-only recordings cannot retroactively supply a fixed global fisheye view. Add synchronized global video during collection, define common transforms, and decide how calibrated virtual-view rendering will coexist with learned camera pose. Changes to the observation interface require retraining or adaptation and new ablations.

11. From a recording to VLA or WAM post-training

A proposed data acceptance and learning pipeline

Validate the recording

Raw clocks, images, pose sources, calibration, contact channels
Detect missing streams, sync failures, drift, jaw errors
Retain rejected episodes and reasons separately

Make robot supervision

TCP frame, feasible trajectories, executed versus desired motion
Held-out sites/operators; success, failure, recovery labels
Versioned VLA / WAM adapters and matched baselines

Test the gain

Complete physical tasks at a fixed adaptation budget
Report latency, resets, interventions, weakest tasks
Scale accepted data only when benefit survives transfer

This is a proposed engineering workflow, not a pipeline already jointly validated by the cited projects

Define the tool-center-point (TCP) transform, pose convention, jaw units, and timestamps before exporting actions. Human handheld motion labels describe what happened to the tool; robot commands describe what the controller should attempt. Compliance and contact can make those quantities diverge. Preserve contact phases, actual tracking, and intervention provenance when mixing handheld and teleoperated data. Neither a file named “action” nor a LeRobot export guarantees equivalent supervision.

For a vision-language-action model (VLA), first compare same-task adaptation with robot-only, handheld-only, and mixed data at matched budgets. For a world-action model (WAM), preserve future video and action alignment, camera identity, and valid contact observations. A better-looking predicted video does not prove improved control. None of the device results reviewed here directly establishes a WAM post-training advantage; that remains an experiment to run.

Split sites, operators, and capture sessions before sampling adjacent clips. Report accepted hours alongside raw hours, and record resampling, dropped frames, pose interpolation, and manual corrections. Evaluate full-task success separately from subtask progress. Training resources should include failed runs and preprocessing; deployment timing should include image age, network queues, policy inference, and actuation. This is where a cheap gripper can turn into an expensive data operation—or a genuinely useful one.

12. What should a robotics team reproduce first?

GoalFirst candidateDecision evidence to collect
Portable baselineSpecialist for lightweight capture; UMI-Live for a coordinated processing workflowAccepted trajectories per operator-hour, label error, processing backlog, release completeness
Data platformYUBI ecosystem and GoSite/task diversity, annotation cost, rights, access control, downstream marginal gain
Precise contactEucalyptus or PolyUMI, according to the missing measurementMatched sensor ablation, contact failures, sensor wear, target-robot sensing
Five-finger tasksAero-UMI as a matched-hand candidateRetargeting to the intended hand, autonomous task results, synchronization verification
Mobile manipulationHoMMI-style head–hand policy and whole-body controlNew-site navigation, reachability, recovery, and active perception

Our first practical experiment would compare Specialist and UMI-Live on the same bimanual task, operator group, and held-out robot evaluation. Count all setup, failed reconstruction, processing, and reset time. This would test whether lighter capture or a more automated pipeline yields more useful robot supervision per unit cost. Public information supports both as serious candidates; it does not establish an overall engineering winner.

Specialist is the most instructive project here for inspecting the capture-to-execution contract: shared morphology, separate labeling and policy views, reconstructible layers, and explicit latency limits. Its next decisive evidence is a complete public learning/control release and task-level trials with failures included. UMI-Live’s next need is broader, controlled evaluation; YUBI’s is the economics and generalization of data scaling; contact and dexterous devices need matched downstream gains rather than richer specifications alone.

UMI is becoming a family of data systems. The useful end point is a verified relationship between what the collector observes, what the human does, and what the robot can execute. Build that relationship first. Then scale the hours, add modalities, or widen the action space when a measured failure gives a reason to do so.

Capture quality is the interface to model scaling

The WAM comparison explains what these devices must export: synchronized observations and actions in an interface the target policy can use. Its evidence that some UMI post-training transfers across embodiments does not make every UMI corpus interchangeable or eliminate same-task robot validation. Specialist is especially useful for inspecting that contract. For contact sensing, the Sharpa survey asks the next question: does the additional signal improve autonomous execution on the receiving robot?

Primary sources and reproducibility references

  1. UMI Arena official accepted devices and papers
  2. Original UMI — interface, relative actions, and latency matching
  3. Specialist official project — hardware, reported experiments, limits
  4. Specialist reconstruction code — factor graph and state layers
  5. Specialist scene setup — anchor scale and visibility
  6. Specialist dataset guide — processing and training-state export
  7. Specialist Insta360 driver — latency method and measurements
  8. Specialist organization — released and pending components
  9. UMI-Live paper v1 — hardware and controlled policy evaluation
  10. UMI-Live official pipeline — calibration and coordinate alignment
  11. UMI-Live dataset card — episode counts and merge provenance
  12. UMI-Live project — physical evaluation and teleoperation setup
  13. YUBI paper — hardware and original corpus
  14. AIRoA-0 report v1 — corpus, partial-credit evaluation, and adaptation
  15. YUBI Go — Quest application, export, and research-software limits
  16. Eucalyptus workshop paper — direct sensing and limitations
  17. YAM-UMI official repository — hand-worn matched interface
  18. PolyUMI / VisTA paper v1 — multimodal policy and sensor ablations
  19. Koala official project — capture/robot co-design
  20. HoMMI official project — head–hand representation and whole-body control
  21. Aero UMI vendor specifications — matched hand and synchronized sensing
  22. Fisheye-VLA paper v1 — observation interface and closed-loop comparisons
  23. CamVLA paper v1 — geometry heads and physical viewpoint shifts

Related: HumanWAM and GR00T 2 data and post-training