Xiaomi Auto World Model Team November 2026

Xiaomi AutoWorld Model 2.0

Causal World Models for Real-World Driving and Embodied Intelligence

Browse all demos

Research highlights

Key features

From physical interactions to controllable worlds and action.

1–7camera views
12.6Mtraining clips
Driving + Roboticsone shared architecture
Technical report

  1. Physical Interaction Data

    General video, driving and robot data, plus rare physical events.

    Data foundation

  2. Unified Views and Controls

    One architecture for 1–7 views, with camera, action and layout control.

    Architecture

  3. Efficient Streaming Generation

    Efficient attention and distillation for fast, stable long rollouts.

    Training recipe

  4. Persistent 3D Scene Context

    A persistent 3D scene from WorldRec 2.0 anchors generation across time and views.

    WorldRec 2.0 framework

  5. Simulation and Policy Learning

    Generated futures for closed-loop evaluation and RL; world representations transfer to action models.

    Results

01 / Data foundation

Broader experience.
Richer physical interactions.

12.6Mtraining clips in the reported corpus

General video Appearance & motion

Web and structured video for appearance, camera geometry and motion priors.

Physical-world data Driving & robotics

Driving logs and robot episodes with trajectories, actions and object states.

Physical-causal data Rare events & interventions

Simulation, generated and public footage covering rare events and interventions.

Shared filtering, captioning and condition annotation.Enlarge figure
Training samples by data sourceEnlarge figure
Caption length by data sourceEnlarge figure
General video scene distributionEnlarge figure
Physical-world driving scene distributionEnlarge figure

Training data and curriculum

StageResolutionViews GeneralDrivingRobotPhysical causal
Samples——3.6M full · 0.5M HQ5.4M3.6M37K engine · 29K LVM · 12K public
Pre-train480p1VFull (includes HQ)YesYesYes
Mid-train480p, 540p1–7VHQ onlyYesYesYes
Post-train480p, 540p1–7V—YesYes—

The 0.5M high-quality subset is part of the full general set.

02 / Capabilities

See the world respond.

Explore a different future.

Long rollouts and edited trajectories in a seven-camera driving scene.

01 / 05

Seven-view generation on a logged trajectory over approximately 45 seconds.

Conditioned on

Initial observations, ego trajectory and scene context.

What to look for

Track road geometry and surrounding traffic as the ego path changes.

03 / Capability comparison

World models at a glance.

Source: report, Table 1 ↗
ModelHorizonViewsControlsDomainApplicationsReal-time
LingBotWorldMinutes1Camera pose, actionGeneralInteraction✓
DreamXWorldMinutes1Camera pose, actionGeneralInteraction✓
CausalWMSeconds1–4ActionRobotSimulation—
HorizonDriveMinutes1LayoutDrivingSimulation—
OmniDreamsMinutes1 / 4 / 7LayoutDrivingSimulation, policy✓
X-World~24 s7LayoutDrivingSimulation✓
Ours 1.0Minutes1 / 3LayoutDrivingSimulation✓
Ours 2.0Minutes1–7Camera pose, action, layoutPhysical AIInteraction, simulation, policy✓

✓ = real-time support; — = not specified.

04 / The approach

A shared architecture.
Many ways to shape the future.

Architecture and training pipeline.Enlarge figure
01

Learn physical evolution

Mix general, driving, robot and rare-event data from pre-training.

02

Unify views and controls

One backbone links views and time.

03

Keep the world grounded

WorldRec 2.0 adds persistent 3D context; efficient attention enables streaming.

Three stages. One continuous foundation.

01 / Pre-train

Build a broad dynamic prior.

Single-view video learning on general and physical data.

1 view480pGeneral + physical data
02 / Mid-train

Make the future controllable.

Add multi-view data plus camera and world-control conditions.

1–7 viewsCamera-pose controlWorld control
03 / Post-train

Stream with fewer denoising steps.

Distillation for faster, stable long rollouts; per-scenario weights.

Few-step distillationLong-video rolloutsScenario-specific weights

WorldRec 2.0: interactive 3D reconstruction.

Static and dynamic 3D Gaussians from multi-camera observations.

WorldRec 1.0: isolated fusion. WorldRec 2.0: interactive query updates.

05 / Measured results

From generation
to decisions.

Generation

PAI-Bench-G · 118 videos

Driving generation

78.0

Highest quality score among the compared driving methods.

  1. Wan2.2-5B76.1
  2. Wan2.2-14B77.0
  3. HunyuanVideo 1.577.4
  4. LingBot-World77.3
  5. Ours78.0

PAI-Bench-G · 174 videos

Robot generation

77.1

Highest quality score among the compared robot methods.

  1. Wan2.2-5B74.3
  2. Wan2.2-14B75.4
  3. HunyuanVideo 1.576.9
  4. LingBot-World77.0
  5. Ours77.1

TriWorldBench · multi-view

Embodied generation

64.79

TWB score; ours leads trajectory accuracy.

Color shows rank within each column.

Embodied generation: all metrics; higher is better. Color indicates relative performance within each column.
MethodTWBTrajectoryNorm. PSNRVQAFlow
DreamDojo51.7216.6159.7847.826.52
WoVR Plus65.3945.2774.6269.1124.85
CausalWM66.0442.9872.4170.0322.99
Ours64.7947.2573.4469.3123.76

Camera controllability · average of two sets

Camera control

78.22

Average score; best on both WorldScore (86.35) and Combined (70.08).

  1. CogVideoX-I2V32.50
  2. Hunyuan 1.534.43
  3. LingBot World39.02
  4. FantasyWorld-1.062.87
  5. DreamXWorld62.12
  6. Ours78.22

Depth · reported dataset average

Depth prediction

96.3

δ1 accuracy; also the lowest AbsRel (0.058) of all methods compared.

  1. Cosmos-4BAbsRel 0.15577.1
  2. Cosmos-12BAbsRel 0.15477.5
  3. Copy Last (VGGT)AbsRel 0.13783.7
  4. AetherAbsRel 0.09687.9
  5. DINO-ForesightAbsRel 0.08291.6
  6. VGGT-WorldAbsRel 0.06594.0
  7. OursAbsRel 0.05896.3

RefSeg IoU · by time offset

Referring segmentation

0.6726RefSeg IoU at t

IoU stays above 0.55 through t + 10 and decays at longer offsets.

  1. t0.6726
  2. t + 50.5934
  3. t + 100.5501
  4. t + 200.3104
  5. t + 300.1705

TriWorldBench: 500 episodes, 50 tasks; external results from the Sept 30, 2026 leaderboard.

Driving generation: full comparisons

MethodViewsFramesFID ↓FVD ↓
ShortVista1256.9089.40
DriveDreamer6814.90340.80
MagicDrive-V261620.9194.84
Ours1 / 62112.09 / 19.62135.13 / 171.73
LongSelf-Forcing121141.53161.00
LongLive121129.05161.41
Causal Forcing++121125.05126.86
Ours1 / 621125.57 / 58.90125.74 / 203.76

Efficiency

Representative attention layer

Attention latency

7.15×Faster attention layer

GLA versus full softmax attention.

Lower is better · ms / layer

  1. Full softmax: 60.14 ms / layer
  2. VTA: 23.17 ms / layer
  3. GLA: 8.41 ms / layer

Causal DiT · per chunk

Generation latency

3.72×Faster causal DiT

GLA versus full softmax attention.

Lower is better · s / chunk

  1. Full softmax: 6.481 s / chunk
  2. VTA: 2.299 s / chunk
  3. GLA: 1.742 s / chunk

Output latent frames

Generation throughput

1.148Latent frames / second

GLA throughput, up from 0.309 with full softmax.

  1. Full softmax: 0.309 latent frames / s
  2. VTA: 0.870 latent frames / s
  3. GLA: 1.148 latent frames / s

FID: 37.24 (VTA) → 38.08 (GLA), +2.3%. Lower is better.

Closed-loop training

0.4%Collision rate

Down from 1.2% for ST-JEPA-E2E.

96.3%Scenario completion

Up from 95.2% for ST-JEPA-E2E.

0.740 sMinimum time to collision

Up from 0.614 seconds for ST-JEPA-E2E.

E2E-WM-RL vs. ST-JEPA-E2E in 1,000 closed-loop simulation scenarios.

Closed-loop policy performance

MethodCR % ↓SCR % ↑OR % ↓TTC s ↑EP % ↑HDR % ↑
DrivoR-E2E1.480.30.00.50481.099.96
ST-JEPA-E2E1.295.20.00.61484.799.93
E2E-WM-RL0.496.30.00.74084.599.99

CR collision rate · SCR scenario completion · OR off-road rate · TTC min. time to collision · EP ego progress (84.5 vs. 84.7) · HDR driving-region accuracy.

World Action Model

NAVSIM v1.1 · navtest

Driving policy

95.3PDMS with world-model conditioning

World-model conditioning adds 0.4 PDMS points versus Base.

Color shows rank within each column.

Driving policy: all metrics; higher is better. Color indicates relative performance within each column.
MethodPDMSNCDACTTCComfortEgo progress
Epona86.297.995.193.899.980.4
DriveZero94.899.099.296.0100.093.1
Base94.999.198.797.099.992.8
Base + causal world model95.399.199.096.699.993.6

LIBERO Plus · task success

Robot policy

75.0%

75.0% weighted average. GAM reports the highest average at 85.5%.

Color shows rank within each column.

Robot policy: all metrics; higher is better. Color indicates relative performance within each column.
MethodAvg.LayoutCameraRobotLanguageLight
Fast-WAMEmb. PT: No49.961.416.544.567.079.9
Fast-WAM-JointEmb. PT: No62.274.827.960.989.386.7
4D-WAMEmb. PT: Yes71.079.245.264.390.694.3
GAMEmb. PT: Yes85.579.183.170.084.897.2
OursEmb. PT: No75.081.539.674.795.397.5

Reconstruction

29.88 dBFull-image PSNR

Up from 28.48 dB on Waymo.

34.3%Lower dynamic depth error

Dynamic-region RMSE: 6.07 → 3.99.

+1.83 dBZero-shot transfer

PSNR gain on nuScenes over WorldRec 1.0.

Novel-view reconstruction · Waymo

MethodDyn. PSNRDyn. D-RMSEFull PSNRFull SSIMFull D-RMSE
STORM22.107.5026.380.7945.48
DGGT22.806.5527.410.8463.47
WorldRec 1.025.016.0728.480.8613.35
WorldRec 2.025.813.9929.880.8652.89

Zero-shot PSNR +0.99 dB on Argoverse 2; after fine-tuning +1.72 dB (nuScenes), +0.97 dB (Argoverse 2).

Xiaomi Auto World Model Team

Explore the full research.

Read technical report Download PDF

Research figure

Open original ↗

Zoom to inspect labels. Scroll to move around the figure. Press Esc to close.

Model demonstration

Use the player’s fullscreen control for a larger view. Landscape orientation helps with multi-camera clips.