DiffWAM

Learning World Action Model from Consecutive Visual Difference

Xinheng Zhang1,2,* · Haoyu Xiong3,4,* · Chao Jing3,5 · Xiaohan Zhang1 · Linshan Li1 · Hongbang Fan3,4 · Xingyu Chen3,†

* Equal contribution.   † Corresponding author.

  1. 1 College of Information Science and Electronic Engineering, Zhejiang University
  2. 2 Polytechnic Institute, Zhejiang University
  3. 3 Zhongguancun Academy
  4. 4 Institute of Automation, Chinese Academy of Sciences
  5. 5 School of Intelligent Robotics and Advanced Manufacturing Innovation, Fudan University

Overview

DiffWAM overview: limitations of VLA, WAM, and WAM-s; complementary future-state and visual-difference modeling; qualitative attention maps; and in-distribution and out-of-distribution success rates.

Learn states and changes

Jointly predict future states and consecutive visual differences.

Train three experts

World, Diff, and Action Experts learn under structured attention.

Keep inference efficient

Deploy only the Observation Encoder and Action Expert.

Method

Three experts learn. One expert acts.

Shared visual layers

World and Diff Experts share early processing.

World Expert

Predict future states.

Diff Expert

Predict consecutive latent differences.

DiffWAM architecture with Observation Encoder, World Expert, Diff Expert, and Action Expert.
Structured attention masks used by DiffWAM during shared and independent expert processing and inference.

Structured attention

Allowed attention

Current observation and each expert's own tokens.

Blocked attention

No attention between different experts.

At inference

Keep the observation and action pathways only.

Training & inference

Scale the optimization strategy with the data.

DiffWAM-S and DiffWAM-L training configurations and their shared inference pathway.
S

DiffWAM-S

Small data. Freeze the Diff Block.

L

DiffWAM-L

Large data. Pretrain, then train jointly.

→

Same inference path

Observation Encoder + Action Expert.

Results

98.8%

LIBERO

+1.75 pts vs. FastWAM

92.65%

RoboTwin 2.0

+1.55 pts vs. FastWAM

88.3%

Real world

+15.0 pts vs. FastWAM

01 Simulation

LIBERO success rate (%)

98.8% average · +1.75 pts vs. FastWAM

MethodSpatialObjectGoalLongAvg.
Motus96.8099.8096.6097.6097.70
Lingbot-VA98.5099.6097.2098.5098.45
FastWAM98.2099.8097.8092.4097.05
ImageWAM97.2099.2098.8098.4098.40
WLA99.00100.0097.8097.6098.60
DiffWAM-S (Ours)98.0099.8099.0098.4098.80

RoboTwin 2.0: Full-data evaluation (%)

92.65% average · +1.55 pts vs. FastWAM

MethodCleanRandomAvg.
ABot-M086.6085.0885.84
Motus88.6687.0287.84
Lingbot-VA92.9091.5092.20
FastWAM*91.8890.3291.10
WLA92.9490.0291.48
DiffWAM-L (Ours)93.2892.0292.65

RoboTwin 2.0: Clean-to-random transfer (%)

MethodEmbodied pretrainingCleanRandomAvg.
π0Yes46.416.331.35
X-VLAYes68.020.944.45
π0.5Yes70.746.058.35
AHA-WAMNo64.33.233.75
FastWAM†No70.21.5035.85
DiffWAM (Ours)No69.75.2737.47

03 Efficiency

Training efficiency

LIBERO optimization steps to 95% success and task-wise success improvements over FastWAM.

Citation

Cite DiffWAM

@article{zhang2026diffwam,
  title   = {DiffWAM: Learning World Action Model from Consecutive Visual Difference},
  author  = {Xinheng Zhang and Haoyu Xiong and Chao Jing and Xiaohan Zhang and Linshan Li and Hongbang Fan and Xingyu Chen},
  journal = {arXiv preprint arXiv:2611.19904},
  year    = {2026},
  eprint  = {2611.19904},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url     = {https://arxiv.org/abs/2611.19904}
}