A robot can produce an attractive point cloud and still fail to return to a chair it passed a minute ago. Geometry does not identify the intended chair, decide when a stage is complete, or guarantee that a commanded turn actually happened. The useful lesson of BEVFormer and UniAD is how to connect spatial evidence to decisions. That lesson gives Pi3X a more productive role in a Compass 1-style navigation system.
Our proposed sequence is to validate metric supervision, build memory that a policy demonstrably uses, then train the interfaces toward complete navigation. Joint future prediction and failure-driven post-training come after those checks. The deliverable after one year should be a defensible decision about a bounded navigation pilot, supported by held-out robot trials and a measured cost ledger.
Evidence checked on 15 November 2026. The planning period is 15 November 2026–14 November 2027. Paper results below are author-reported; architecture proposals, budgets, and acceptance thresholds are 1N Robotics research recommendations, not achieved results or product commitments. Pi3X has not been validated here as a replacement for Compass 1’s Pi3 + MoGe-2 pipeline.
1. BEVFormer to UniAD: change what the representation is trained to serve
BEVFormer converts calibrated surrounding-camera features into a bird’s-eye-view (BEV) grid. Spatial cross-attention gathers evidence around projected 3D reference points, while temporal self-attention fuses historical BEV features with ego-motion alignment. This creates a shared spatial memory for detection and map segmentation. It is a learned feature representation; the method does not need a complete photorealistic reconstruction before it can perceive the scene. [1]
UniAD asks whether intermediate tasks help the final plan. Track queries carry agent identity, map queries encode road structure, motion queries model possible futures, and occupancy prediction supports trajectory refinement. These learned interfaces connect tasks whose losses are optimized together. The original training first fits perception components, then trains the full stack. End-to-end learning still contains deliberate structure and supervision. [2]
| Design question | BEVFormer | UniAD | Transfer to robot navigation |
|---|---|---|---|
| What is shared? | Calibrated spatial grid and temporal features | Task queries and planning-relevant features | Metric frame, target identity, history IDs, and timing |
| What is optimized? | Perception outputs | Coordinated perception, prediction, and planning | Complete navigation plus useful auxiliary tasks |
| What should be retained? | Multi-view alignment and temporal persistence | Structured interfaces and staged joint training | Observable state and debuggable action interfaces |
| What should not be copied blindly? | A fixed road-plane grid | Every automotive task head and loss | Use only geometry and tasks needed by the deployment domain |
For a robot, useful auxiliary outputs may be a target’s relative bearing, a traversable local corridor, or whether an intended turn was completed. A larger collection of heads also costs labels and can create conflicting gradients. Our interpretation of the UniAD lesson is to measure each head’s contribution to action and recovery, rather than treating multi-task learning as an automatic improvement.
2. Pi3X and Compass 1 solve different parts of the problem
Pi3X predicts camera geometry and point maps with approximate metric scale, and supports optional intrinsics, depths, and poses. Its implementation applies a predicted metric factor to reconstructed points and camera translation. This makes it a candidate geometry teacher and annotation tool. A visually smooth reconstruction is still an estimate: scale, dynamic objects, stabilization, and sliding-window alignment can corrupt the small motion increments needed for navigation. [3] [4]
Compass 1 uses Pi3 + MoGe-2 offline to derive action labels. Online, sparse keyframes and a History Representation Map (HRM) feed a Cosmos3-Edge-4B backbone with structured visual question answering (VQA) and joint video-action generation. HRM is a canvas of traveled positions, headings, and keyframe IDs, not a dense 3D obstacle map. The robot additionally relies on odometry-backed execution. Replacing its offline geometry tool with Pi3X would change the supervision pipeline, not by itself create a new navigation policy. [5]
Driving: representation → decision
Robot data: video → verified labels
Proposed policy: memory → execution
Pi3X → Compass 1 is a proposed engineering path, not a published model lineage or validated combined system
3. The evidence supports a hypothesis, not a combined-system score
| Published evidence | Reported result and setting | What remains unproven |
|---|---|---|
| BEVFormer v2 | 56.9 NDS, nuScenes test, V2-99 with extra depth pretraining | Detection quality does not measure mission success |
| UniAD v2 versus ST-P3 | Open-loop mean planning L2: 2.11 → 1.03 m; collision metric: 0.71% → 0.31% | Not physical collision frequency or long-run autonomy |
| Compass 1 main Go2 benchmark | 254/320 real trials, 79.4%, four scenarios | Not a Pi3X evaluation or four-embodiment pooled result |
| Compass 1 matched ablations | 160 trials/variant, Office + Street: full 78.8%; no HRM 53.1%; no video 46.9% | Neither ablation removes offline Pi3 + MoGe-2 reconstruction |
Sources: the original BEVFormer and UniAD tables, and Compass 1’s paper and official evaluation. No shared protocol ranks these four methods, and no reported result isolates a Pi3X substitution. Our testable hypothesis is narrower: better calibrated motion labels and explicitly supervised spatial memory can improve navigation at matched data and inference budgets. Both parts must earn their place in physical trials. [1] [2] [5] [6]
4. Build a geometry-informed policy with a separate execution contract
For the first year, assume one low-speed wheeled platform, single-floor navigation, three capability groups—following staged instructions, finding an initially unseen target, and returning to a visited landmark—and instrumented development sites. Start with RGB and odometry; use a temporary reference sensor or surveyed markers to validate labels. Camera height transfer and a second embodiment are later tests. This scope is a proposed research starting point, not a description of 1N Robotics’ shipped hardware or a customer commitment.
Keep Pi3X offline initially. Retain frame timestamps, camera intrinsics and mounting transform, stabilization mode, reference trajectory, model version, and quality flags. Transform camera motion into the robot frame before producing action labels; a camera turn and a body turn are different when the mount moves. Compare image-only Pi3X with Pi3 + MoGe-2 and MapAnything on the same videos, with identical sampling and independent reference measurements. Do not use free scale alignment when scoring metric accuracy. [3] [4] [7]
Online, keep two memories distinct. A short-lived local geometry buffer records observed free space, obstacles, timestamps, and uncertainty. A longer-lived topological/visual memory records landmarks, keyframes, and route connections. Unobserved space remains unknown; a missing point is not evidence of free space. Expired dynamic-object evidence must not become a permanent obstacle, and a historical target location must not imply the target is still there. Pose corrections should update both memories or invalidate inconsistent entries.
Begin with a Compass 1-style HRM interface and a direct action policy. Once memory helps, test explicit target, stage, and corridor queries that attend to the spatial buffer, inspired by UniAD’s interfaces. Supervise them with measured or verified labels and a task-relevant loss, then test joint optimization. Keep the existing HRM-only policy as a fallback. This proposed query interface changes the architecture and needs retraining; it is not a free upgrade to a released Compass checkpoint.
Offline supervision
Online navigation policy
Independent execution layer
The geometry cache, query interface, and acknowledgment loop are proposed additions; no joint-system result is claimed
How this plan connects to the rest of the research
The driving analysis supplies the representation-to-decision principle; the Compass 1 study supplies a concrete data and memory recipe. Together they motivate the plan, but do not validate its proposed combination. Agentic recovery is a later interface experiment, after basic execution and stop behavior work. Count retries, human resets, and total wall time so a better agent score cannot hide a weaker navigation policy.
5. Train in stages and isolate the source of every gain
| Experiment | Change one factor | Hold fixed / acceptance question |
|---|---|---|
| E1: geometry labels | Reference motion versus Pi3X labels; Pi3 + MoGe-2 and MapAnything as offline controls | Same clips and policy; does accepted-label yield improve without corrupting motion? |
| E2: memory | Recent frames → indexed keyframes → HRM + retrieval supervision | Same action labels and backbone; does held-out return success improve? |
| E3: planning-oriented interfaces | Add target / stage / corridor auxiliary queries individually, then jointly | Same training samples and update budget; which auxiliary output helps action? |
| E4: future prediction | Direct action versus joint video-action training | Same geometry, memory, data exposure, and runtime allowance; is added cost justified? |
| E5: post-training | Failure/recovery SFT, then tightly scoped simulator RL if warranted | Match recovery permissions and evaluation resets; does real transfer improve? |
Run these experiments sequentially, not as an immediate large factorial sweep. Carry the winning version forward, reserve a development set for model selection, and freeze a final test set. First validate labels and a supervised policy, then warm up new query heads before unfreezing a limited backbone region. Parameter-efficient adaptation is a candidate cost control, not a published Compass 1 recipe. Report trainable parameters, all tested runs, accelerator-hours, and annotation labor; retain failed runs in the ledger.
Compass 1 reports full end-to-end co-training on 32 H200 GPUs for 60,000 steps, batch 256, with VQA/video/action weights 3:1:1. That is evidence about its recipe, not a mandatory starting budget. Our initial baseline keeps video generation optional. A later geometry-aware objective could combine action imitation, stage/stop prediction, target retrieval, and confidence-masked geometry consistency. Tune weights against physical task results, and do not equate token accuracy or reconstruction loss with successful navigation. [5]
Split data by physical site, route, capture session, and target instance before deriving overlapping windows. Keep ordinary demonstrations, near-failures, human corrections, recovery, and forced stops distinguishable. Every accepted training example needs provenance back to the raw frames and quality decisions. A small set of reference-quality motion is still needed even if most examples come from inexpensive human video. Report rejected hours and engineering time per accepted hour; raw video duration alone obscures the economics.
In the proposed joint-training phase, keep the offline geometry teacher frozen. Train the student’s perception, memory, and action interfaces together; auxiliary geometry targets should be masked by independently calibrated quality rather than assuming the teacher’s confidence is a probability of correctness. Joint training here does not mean differentiating through Pi3X label production, the physical robot, or an unverified simulator. If geometry loss improves while navigation worsens, reduce or remove that objective.
6. A twelve-month plan with explicit decision gates
The dates below are a proposed schedule. A small team needs ownership of geometry/data quality, model training/evaluation, and robot execution/integration; these are roles, not an assumed staffing commitment. Start with a 20-hour curated pilot corpus and roughly 200 independently checked motion segments, as planning caps rather than sample-efficiency claims. Profile storage, labeling time, and one training run before approving any expansion. A missed gate delays downstream work or selects the simpler alternative.
| Period | Deliverable | Proposed gate / fallback |
|---|---|---|
| M1–2 2026-11-15 2027-01-14 | Synchronized capture, reference trajectories, frozen splits, reconstruction audit | Initial target: median segment-scale error ≤5%, p95 ≤15%; otherwise anchor labels with reference motion and repair calibration |
| M3–4 2027-01-15 2027-03-14 | Direct-action baseline and indexed-memory / HRM variants | Matched development trials should show ≥10 percentage-point return-task gain without worse interventions; otherwise retain the simpler memory |
| M5–6 2027-03-15 2027-05-14 | Target/stage/corridor query ablations and staged joint training | Seek ≥5 percentage-point complete-mission gain at matched runtime; otherwise keep HRM-only interfaces |
| M7–9 2027-05-15 2027-08-14 | Video-action versus direct-action study; failure/recovery data and optional narrow RL | Add prediction only if it improves task/recovery results within measured latency and cost caps; abandon RL if simulator gains do not transfer |
| M10–12 2027-08-15 2027-11-14 | Frozen policy, untouched-site evaluation, repeated operation, and pilot decision | Candidate target: ≥90% first-attempt complete success over 200 preregistered missions, with per-task reporting and bounded interventions; otherwise narrow scope or continue research |
All thresholds in the table are proposed project gates, not published measurements or general safety standards. Define the scale-error segment length, minimum translation, time horizon, and reference uncertainty in M1; near-zero motion makes a percentage unstable. Fix gate definitions after that pilot and before comparing candidates. The final 200 missions should be stratified across capability groups and untouched sites, include stopping and recovery requirements, and count a human takeover as a first-attempt failure. Report uncertainty clustered by route/site when trials share conditions.
A 90% point estimate alone is insufficient for a pilot. Also publish the weakest task group, completion time, intervention causes, safety events, and throughput. Add repeated two-hour sessions across days as an initial endurance check; two hours without incident is not a safety certification. A cross-camera or second-platform test must state whether weights, calibration, the motion adapter, or the low-level controller changed. Only unchanged weights justify calling the policy transfer zero-shot, and even then engineering adaptation must be disclosed.
7. Latency and cost determine the deployment architecture
Compass 1 reports 0.43–0.71 seconds to produce twelve actions on a remote PRO 6000. Its quoted 28–17 Hz is action throughput, not a fresh visual decision at that rate. A robot can travel while a chunk is being computed or executed. Timestamp every observation and command, profile p50/p95/p99 sensor-to-actuator age with queueing and network delay, and discard stale chunks. A model forward-pass timer cannot establish this contract. [5] [6]
Set the latency limit from the operating speed, observed clearance, and measured braking ability. A first approximation is stopping distance ≥ vτ + v²/(2a) + margin, where τ includes sensing, communication, decision, and actuation delay, and a is a conservative measured deceleration. This is an engineering budgeting relation, not a certified safety model; people, slip, slopes, and localization error need additional treatment. Keep local stop/clearance checks independent of remote language inference, and slow the platform when freshness cannot be guaranteed.
| Cost item | Measure before scaling | Decision it informs |
|---|---|---|
| Geometry and annotation | Accepted hours / engineer-day; scale failures; human corrections; rejected video | Pi3X teacher, reference-label pipeline, or alternative reconstruction model |
| Training | All accelerator-hours, trainable parameters, peak memory, and unsuccessful runs | Small student / parameter-efficient adaptation versus full joint model |
| Runtime | End-to-end command age, energy, bandwidth, continuous-run cost, reset labor | Local deployment, split execution, or simpler policy |
Do not train a new foundation model from scratch for this first-year plan. Profile available pretrained models and a small student first, using a fixed per-experiment compute allowance established in M1. Pi3X’s public repository distinguishes permissive code from non-commercial model weights; Compass deployment code and checkpoints remain scheduled for December 2026 at this check. Check the relevant weight, training-data, and generated-label reuse terms before a commercial pilot. If releases or rights are unavailable, continue the interface experiments using an available baseline with suitable rights rather than making the schedule depend on a future download. [3] [8]
8. What would justify the next year of work?
The proposed progression is geometry that can be audited, memory that changes behavior, and joint training that improves complete tasks. BEVFormer contributes spatial and temporal organization; UniAD contributes planning-oriented interfaces and staged optimization; Pi3X contributes a candidate source of metric supervision; Compass 1 contributes a recipe for using history in navigation. Their roles are complementary, and none of their individual benchmark scores predicts the performance of the proposed combination.
The strongest outcome by November 2027 would be a bounded pilot candidate that completes new-site missions with fewer interventions at a measured latency and cost. A useful negative outcome would be discovering that verified odometry labels and a simple HRM policy outperform a geometry-heavy joint model under the same budget. Both outcomes settle an engineering question. The plan should stop rewarding point-cloud appearance, language plausibility, or video quality when those improvements do not survive physical execution.
This remains a research-to-pilot roadmap. It does not establish all-weather autonomy, human-safe operation, or scalable deployment. The first-year bet is that spatial supervision and memory can reduce the number of new robot demonstrations needed for a reliable bounded task. The milestones are designed to find out whether that bet is true before committing to a larger model or data program.
Primary sources and companion analyses
- BEVFormer v2 — architecture and original evaluation
- UniAD v2 — task queries, staged training, and planning table
- Pi3 / Pi3X official repository — capabilities and licenses
- Pi3X implementation — metric factor and camera outputs
- Compass 1 paper v1 — architecture, labeling, training, and physical tests
- Compass 1 official research page — evaluation and ablation protocol
- MapAnything official repository — optional geometric inputs
- Compass release repository — current release schedule
Companion: driving representations and interactive data engines · Companion: Compass 1 camera, geometry, and memory