DreamerV3 — one set of hyperparameters works across 150+ tasks, including collecting diamonds in Minecraft from scratch.DreamerV3：一套超参数适用于 150 多个任务，包括从零开始在《我的世界》中挖到钻石。
@misc{hafner2023mastering,
title = {{Mastering Diverse Domains through World Models}},
author = {Danijar Hafner and Jurgis Pasukonis and Jimmy Ba and Timothy Lillicrap},
year = {2023},
eprint = {2301.04104},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2301.04104},
}
Adds a 257M latent world model to a pretrained VLA so predicted future features condition the action expert, with no pixel generation or video pretraining; reports 98.7% on LIBERO and real-robot gains over π0.5.在预训练 VLA 上加一个 2.57 亿参数的潜空间世界模型，用预测的未来表征去条件化动作专家，无需像素生成或大规模视频预训练；报告 LIBERO 成功率 98.7%，真机任务上明显优于 π0.5。
@misc{wang2026wave,
title = {{WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction}},
author = {Haomin Wang and Anqiong Ji and Ke Wang and Zhongshan Liu and Ru Ying and Ran Zheng and Zhipeng Zhang},
year = {2026},
eprint = {2610.18568},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.18568},
}
Between direct action prediction and full video prediction: VLDA learns a compact latent 'Delta' of task-relevant state change and uses it to seed action denoising. Built on Baidu's open LoongForge.介于直接预测动作与完整预测未来画面之间：VLDA 学习一个紧凑的隐变量 Delta 表示与任务相关的状态变化，并用它引导动作去噪；实现于百度开源训练框架 LoongForge。
@misc{zhang2026vision,
title = {{Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation}},
author = {Lingsen Zhang and Wei Li and Jie He and Henghua Zhang and Kai Mo and Shiyong Li and Yanpeng Wang and Rui Shao},
year = {2026},
eprint = {2610.25874},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.25874},
}
Splits the VLA vs WAM debate into backbone, future loss and inference machinery: a foresight loss predicts frozen future features, no video generation. Reports beating FastWAM on LIBERO-Plus (64.5% vs 55.3%).把 VLA 与世界动作模型（WAM）之争拆成骨干初始化、未来预测损失和推理时是否保留预测模块三个选择：用预测冻结未来特征的前瞻损失，训练与推理都不生成视频；报告在 LIBERO-Plus 上 64.5%，FastWAM 为 55.3%。
@misc{yu2026vision,
title = {{Vision-Language-Action or World-Action? Just Action Models}},
author = {Yonghao Yu and Lang Huang and Runyi Li and Toshihiko Yamasaki},
year = {2026},
eprint = {2610.37219},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.37219},
}
VLA that reasons in discrete motion tokens (a chain of dynamics) shared across embodiments; predicted futures are scored against what actually happened, so action-free human videos give RL signal. Sim and real tests.VLA 用跨本体共享的离散运动 token 链（Chain-of-Dynamics）表达动作思考；预测每条链导致的未来并与真实结果比对来强化，使没有动作标注的人类视频也能提供监督。在仿真与真机上验证人到机器人的迁移。
@misc{kim2026thinkworld,
title = {{ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning}},
author = {Chaehyun Kim and Chi-Pin Huang and Jisoo Kim and Min-Hung Chen and Yu-Chiang Frank Wang and Seungryong Kim and Fu-En Yang},
year = {2026},
eprint = {2610.37666},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37666},
}
@misc{wang2026pdrvla,
title = {{PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations}},
author = {Hanqing Wang and Zhaotian Cai and Ruichuan An and Bohan Zeng and Tao Zhang and Yilin Chen and Xu Jiang and Kaiquan Chen and Chongyang Wu and Haoran Li and Fan Yang and Mingan Lin and Wentao Zhang and Zenan Zhou and Youzheng Wu},
year = {2026},
eprint = {2610.37381},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37381},
}
Visuo-tactile world model trained only on humans handling GelSight sensors (4.9 h React dataset). Reports that contact dynamics transfer to robot rollouts more reliably than visual appearance does.只用人手持 GelSight 传感器的交互数据（4.9 小时 React 数据集）训练视触觉世界模型。报告在机器人回放中接触动态比视觉外观更容易迁移，部分任务的相机视角预测甚至不如直接复制帧。
@misc{ma2026acts,
title = {{ACTS: Learning Visuo-Tactile World Models from Human Physical Interaction}},
author = {Yuxiang Ma and Weihan Xu and Pengfei Ye and Anna Mei and Yuxin Ray Song and Tingle Li and Haonan Chen and Edward H. Adelson},
year = {2026},
eprint = {2610.24285},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.24285},
}
Rather than distilling per-motion experts or brute-forcing one RL policy, trains a humanoid tracking policy jointly with one differentiable world model of shared dynamics, fit from simulator logs (CoRL 2026 oral).SuperMimic 不再蒸馏逐动作专家或用大算力硬训单一 RL 策略，而是把人形机器人动作跟踪策略与一个从仿真日志拟合的、共享动力学的可微世界模型联合训练（CoRL 2026 口头报告）。
@misc{liao2026supermimic,
title = {{SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model}},
author = {Qiayuan Liao and Yiyang Shao and Junfeng Long and Kevin Zakka and Roberto Horowitz and Pieter Abbeel and Koushil Sreenath},
year = {2026},
eprint = {2610.28480},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28480},
}
An egocentric world model covering both walking and manipulating, trained on ~10k hours of human video plus ~3k hours of robot data under one shared kinematic action representation.同时覆盖移动与操作的第一人称世界模型，在约 1 万小时人类视频和约 3 千小时机器人数据上训练，用统一的运动学表示描述不同本体的动作。
@misc{liu2026ropedia,
title = {{Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience}},
author = {Tianqi Liu and Hanfei Zhao and Qifeng Liang and Hui En Pang and Jiarui Xu and Sayang Mu and Penghao Qiu and YingSheng Zhang and Jun Wang and Ziang Cao and Mingze Sun and Yukang Cao and Fangzhou Hong and Zhaoxi Chen and Ziwei Liu},
year = {2026},
eprint = {2610.26855},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.26855},
}
World Models世界模型egocentrichuman-datacross-embodimentloco-manipulation
A survey of video world models in robotics - where they replace or complement physics simulators (e.g. deformable objects), their applications, and open challenges - with a maintained paper list.机器人视频世界模型综述：它们在哪些场景可替代或补充物理仿真器（如可变形物体）、主要应用与开放挑战，并附持续更新的论文列表。
@misc{mei2026robotic,
title = {{Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directions}},
author = {Zhiting Mei and Tenny Yin and Ola Shorinwa and Apurva Badithela and Zhonghe Zheng and Joseph Bruno and Madison Bland and Lihan Zha and Asher Hancock and Jaime Fernández Fisac and Philip Dames and Anirudha Majumdar},
year = {2026},
eprint = {2601.07823},
archivePrefix = {arXiv},
primaryClass = {eess.SY},
url = {https://arxiv.org/abs/2601.07823},
}
World action model that jointly predicts video, motion and contact wrench, outputting time-aligned motion–wrench chunks as control references (hybrid motion-force style) for force-sensitive manipulation.一个同时预测未来视频、运动与接触力旋量的世界动作模型，借鉴力位混合控制思路，输出时间对齐的运动-力旋量片段作为控制参考，面向仅靠视觉无法感知接触与负载的力敏感操作任务。
@misc{ding2026cw,
title = {{CW-WAM: A Structured Contact-Wrench World Action Model for Force-Sensitive Manipulation}},
author = {Houzhu Ding and Xinda Qi and Luqi Wang and Ming Yang},
year = {2026},
eprint = {2610.30117},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30117},
}
World action model that brings scene-level 3D flow (tracked 3D points) inside the policy rather than as a separate module or extra head, so 4D geometry and visual semantics are modelled together with actions.世界动作模型：把场景级三维流（跟踪的三维点）纳入策略内部，而不是作为独立模块或额外输出头，让 4D 几何、视觉语义与动作一起建模。
@misc{tian2026uniflowwam,
title = {{UniFlowWAM: A Unified World-Action Model with Visual Semantics and 4D Geometry}},
author = {Jialin Tian and Zhuofan Li and Xinkai Wang and Bokai Lin and Chenyi Wang and Xinyu Zhan and Cewu Lu and Lixin Yang},
year = {2026},
eprint = {2610.30991},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30991},
}
One world model that is also the policy, across 20 robot families mapped to a shared joint-space action format; reports strong WorldArena, RoboTwin 2.0 and LIBERO-plus results and a one-step distilled humanoid policy.同一个世界模型兼作策略，用共享的关节坐标动作空间覆盖 20 类机器人；论文报告其在 WorldArena、RoboTwin 2.0、LIBERO-plus 上表现突出，并蒸馏出可实时运行的单步人形机器人策略。
@misc{sidorov2026greenwam,
title = {{GreenWAM: A World Foundation Model for Robotics}},
author = {Gennady Sidorov and Valerii Kornilov and Denis Grankin and Polina Fedotova and Aleksey Postnikov},
year = {2026},
eprint = {2610.18774},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.18774},
}
FRED trains a world action model to predict better one-step future video features via self-distillation plus history context; reports large gains on long-horizon, out-of-domain and human-video-only tasks.FRED 通过自蒸馏并引入历史观测，让世界动作模型在单步去噪中预测出更好的未来视频表征；报告在长时程、域外环境以及只有人类视频示范的任务上提升明显。
@misc{jang2026self,
title = {{Self-Distilling Future Video Representations for World Action Models}},
author = {Huiwon Jang and Suhyeok Jang and Hyeonmo Kang and Seungku Kim and Jinwoo Shin and Sihyun Yu},
year = {2026},
eprint = {2610.30320},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30320},
}
World Models世界模型world-action-modelvideo-diffusionself-distillationhuman-to-robotreal
AW-0 jointly generates code, future video and actions in one mixture-of-transformers; code structures planning and becomes reusable skills. Reports gains over VLA and WAM baselines on RoboTwin 2.0 and real two-arm tasks.AW-0 在同一个混合 Transformer 中联合生成代码、未来视频和机器人动作三路输出：代码既组织多步预测与控制，又沉淀为可复用技能库；报告在 RoboTwin 2.0 和真机双臂任务上优于 VLA 与 WAM 基线。
@misc{liu2026agentic,
title = {{Agentic World Action Models}},
author = {Zhuoyang Liu and Dantong Niu and Botong Zhang and Boning Shao and Jing Wang and Ruijie Zheng and Yuvan Sharma and Runyu Lu and Jiaming Liu and Renrui Zhang and Roei Herzig and Yutong Bai and Ken Goldberg and Danfei Xu and Yuke Zhu and Guanzhi Wang and Linxi "Jim" Fan and Trevor Darrell},
year = {2026},
eprint = {2610.30136},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30136},
}
SkeLAM splits latent actions into a robot-skeleton-grounded kinematic part and a visual residual, so world-model rollouts follow robot controls; on DROID reports 19.5% lower end-effector error than DreamDojo.SkeLAM 把潜在动作拆成由机器人骨架监督的运动学部分和保留其余视觉动态的残差部分，使世界模型生成更贴合机器人控制指令；报告在 DROID 上末端轨迹误差比 DreamDojo 低 19.5%，预测成功率与真机评测一致。
@misc{lv2026skeleton,
title = {{Skeleton-Grounded Latent Action Model for Robot World Models}},
author = {Qi Lv and Jiayi Lin and Yuxin Jiang and Jianzong Wang and Xiang Deng and Mike Shou Zheng},
year = {2026},
eprint = {2610.31474},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31474},
}
World Models世界模型latent-actionvideo-world-modeldroidpolicy-evaluation
Paper论文 2026 · Alibaba DAMO Academy new新 2026-10-26
A 16B model interleaving video prediction and language reasoning for embodied understanding and long-horizon visual planning, plus a world-action extension for control; reports 51.5% on 20 RoboCasa365 tasks.16B 统一具身模型，交替进行视频预测与语言推理，兼顾具身理解与长程视觉规划，并扩展出用于机器人控制的世界-动作版本；另提出 Rynn-VP、Rynn-GEU 评测，报告在 20 个 RoboCasa365 组合任务上成功率 51.5%。
@misc{zhu2026rynnunify,
title = {{RynnUnify: Towards Unified Embodied Brain}},
author = {Minghao Zhu and Bohan Hou and Kun Xiang and Zixuan Wang and Yinzhou Tang and Yifei Yang and Zhikai Wang and Tianyi Zhang and Kehan Li and Haoyu Zhao and Siteng Huang and Leigang Qu and Sicong Leng and Dongchi Huang and Mingxiu Chen and Shijian Lu and Yue Wang and Xiaodan Liang and Zhenyu Zeng and Deli Zhao and Chengju Liu and Jianfei Yang and Xin Li},
year = {2026},
eprint = {2610.32965},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.32965},
}
Action-conditioned world model that predicts multi-view RGB and tactile images together in one video space, post-trained for state consistency; used for closed-loop policy evaluation and GRPO updates.动作条件的视触觉世界模型：在统一的视频空间中同时预测多视角 RGB 与触觉图像，并经一致性后训练保证与真实任务状态吻合；可用于闭环策略评估和 GRPO 策略更新。
@misc{xing2026physical,
title = {{Physical Canvas: A Vision Tactile World Model For Robot Policy Evaluation And Learning}},
author = {Zebin Xing and Yuhang Zheng and Yupeng Zheng and Yichen Zhang and Zichen Li and Shangqing Zhou and Yujie Zang and Yuchen Liu and Zhongpu Xia and Haoran Li and Qichao Zhang},
year = {2026},
eprint = {2610.28613},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28613},
}
Auxiliary training task for world action models: an EMA teacher sees the clean frame, the online model sees a perturbed one and aligns its action features. Reports better robustness for four WAMs.为世界-动作模型加一个自监督辅助任务：EMA 教师看原始画面，在线模型看外观扰动后的画面并对齐其动作表征；报告在 RoboTwin 2.0、LIBERO-Plus 等基准上提升四种 WAM 对视觉扰动的鲁棒性，且不损失原有性能。
@misc{zou2026pica,
title = {{PICA-WAM: Visual Perturbation Invariance via Cross-Appearance Self-Alignment in World Action Models}},
author = {Yude Zou and Yifei Yao and Guanbo Jiao and Jianming Ma and Guanjie Zheng and Xing Gao},
year = {2026},
eprint = {2610.26813},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.26813},
}
World action model that acts in one diffusion step: 'imagination tokens' replace random-noise initialisation as an action prior. Reports ~3× speedup over direct-policy WAMs, 40 ms latency on an RTX 5090.单步扩散即可生成动作的世界动作模型：用“想象 token”代替随机噪声初始化，为动作提供先验。报告比直接策略式 WAM 快约 3 倍，编译后在 RTX 5090 上延迟低至 40 毫秒，覆盖自动驾驶与具身控制。
@misc{zhang2026oswam,
title = {{OSWAM: An Effective World Action Model with One-Step Action Generation}},
author = {Guowen Zhang and Liyi Chen and Xu Hu and Pengfei Wang and Jingfan Chen and Chenhang He and Lei Zhang},
year = {2026},
eprint = {2610.35850},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.35850},
}
Train the video world model's autoencoder for temporal predictability, not just reconstruction, with a past-conditioned decoder. Reports up to 5× cheaper inference, 7× faster training and lower long-horizon / OOD error.视频世界模型的潜空间不应只为重建而训练：加入时间可预测性目标并配合以历史为条件的解码器。报告推理成本最多降低 5 倍、训练快 7 倍以上，长时域与分布外误差降低 15–24%，与真实策略评测更一致。
Dexterous-manipulation world model that first predicts object motion and deformation in metric 3D from a hand trajectory, then renders video. Reports 27–40% lower moved-point error than PointWorld.灵巧操作世界模型：先根据手部轨迹在度量 3D 空间中显式预测物体运动与形变，再据此渲染视频。报告在 HOT3D、OakInk2、PhysTwin 上运动点误差比 PointWorld 低 27–40%，两步因果生成大幅提速。
@misc{lee2026dexplicit,
title = {{DeXplicit: Making Physical State Explicit in Dexterous Manipulation World Models}},
author = {Adam Lee and Mutian Xu and Yukang Cao and Shulin Tian and Ziwei Liu},
year = {2026},
eprint = {2610.30781},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.30781},
}
Why action-conditioned video models ignore actions: expert demos carry too little action influence. AIM actively collects data to raise it; 500 such episodes + 3k demos match a 300k-demo model on controllability.解释动作条件视频世界模型为何忽略动作：专家示教的“动作影响力”太低，模型容量再大也无法弥补。AIM 主动采集数据提高这一指标；报告 500 条主动采集数据加 3000 条示教即可在末端可控性上追平 30 万条专家数据。
@misc{bateman2026training,
title = {{Training Controllable World Models via Action Influence Maximization}},
author = {Samuel M. Bateman and Tenny Yin and Chongyi Zheng and Lei Huang and Brian Wang and Benjamin Eysenbach and Jaime Fernández Fisac and Dhruv Shah},
year = {2026},
eprint = {2610.27387},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.27387},
}
World action model for moving targets: forecasts the object's future masks, steers a video model with a Trajectory ControlNet, and distills the pipeline to one-step video-action inference to cut latency.面向动态目标的世界动作模型：先预测目标物体的未来掩码轨迹，用 Trajectory ControlNet 引导视频模型生成与轨迹一致的未来，再通过解耦蒸馏实现一步视频-动作推理以降低延迟。
@misc{geng2026trajwam,
title = {{TrajWAM: An Efficient Trajectory-Guided World Action Model for Dynamic Robotic Manipulation}},
author = {Haohan Geng and Sha Zhang and Weishuo Sun and Guanting Ye and Wenhao Yu and Yuang Xie and Qicheng Zeng and Peng Ye and Jiajun Deng and Jianmin Ji},
year = {2026},
eprint = {2610.25763},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.25763},
}
Compact (<100M) world-action model: predicts frozen-DINOv3 tokens at two future horizons and decodes action chunks, with no robot or video pretraining. Reports beating billion-parameter WAMs on RoboTwin 2.0.参数不到 1 亿的紧凑世界动作模型：在冻结 DINOv3 特征空间中预测两个未来时刻的 token 并直接回归动作块，无需机器人或视频预训练；报告在 RoboTwin 2.0 上超过十亿参数级 WAM。
@misc{liu2026gawm,
title = {{GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation}},
author = {Yang Liu and Xiaoli Liu and Liuzhao Li and Yuduo Song and Muyuan Liu and Xiang Gao},
year = {2026},
eprint = {2610.37091},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37091},
}
Turns a pretrained video diffusion transformer into a policy by encoding each action step as one patch in the video latent space; at test time it outputs an action chunk in one DDIM step (84 ms), no video generation.把预训练视频扩散 Transformer 改造成策略：每个时间步的动作编码为视频潜空间中的一个 patch，联合训练视频、视频+动作与纯动作三种扩散目标；测试时一步 DDIM 生成动作块（84 毫秒），无需生成视频。
@misc{collins2026tapestry,
title = {{TAPESTRY: A Single Backbone for Video Generation and Robot Control}},
author = {Jeremy A. Collins and Seungjae Lee and Rachanon Wachakorn and Krishnan Srinivasan and Animesh Garg and Vitor Campagnolo Guizilini and Paarth Shah},
year = {2026},
eprint = {2610.36823},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36823},
}
Cross-embodiment video world model conditioned on RynnLAM's 3D latent actions, pretrained on 23 datasets (~553M frames) of human, robot and sim video; its generated data improves LIBERO and real-robot policies.以 RynnLAM 的三维潜在动作为统一条件接口的跨本体视频世界模型，在 23 个数据集（约 5.5 亿帧）的人类、机器人和仿真视频上预训练；生成的数据提升了 LIBERO 与真机策略成功率。
@misc{zhao2026rynnworld,
title = {{RynnWorld-Latent: A Cross-Embodiment World Model Conditioned on Universal Latent Actions}},
author = {Haoyu Zhao and Yinzhou Tang and Zixuan Wang and Minghao Zhu and Dongchi Huang and Xingyue Zhao and Qin Zhao and Kehan Li and Siteng Huang and Xin Li and Zhenyu Zeng and Mingxiu Chen and Deli Zhao and Zhongyu Li},
year = {2026},
eprint = {2610.36791},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36791},
}
Rewinds an action-conditioned world model to key decision points to try alternative actions, turning outcomes into action-level supervision; real data keeps correcting the model. Reports +10.7–11.9 pts on 3 real tasks.把动作条件世界模型当作反事实引擎：回到关键决策点尝试其他动作并对比结果，得到动作级监督而无需单独的价值模型；真机经验再通过 LoRA 持续修正世界模型。报告在 3 个真实任务上比最强基线高 10.7–11.9 个百分点。
@misc{luo2026rollback,
title = {{Rollback: Experience Backtracking in World Models for Policy Improvement}},
author = {Jiayi Luo and Yang Zhang and Liliang Chen and Nan Wang and Chen Gao and Boxiang Qiu and Zhuoxuan Li and Baorui Peng and Chuankang Li and Yuxiang Yan and Si Liu and Qingyun Sun and Shuicheng Yan and Maoqing Yao and Jianxin Li and Guanghui Ren},
year = {2026},
eprint = {2610.36068},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36068},
}
Adds future 3D point-motion prediction as an auxiliary task to a world action model (video/motion/action MoT); inference stays one visual pass. Reports gains over Fast-WAM with few demos and under shift.在世界动作模型中加入未来三维点运动预测作为辅助目标（视频/运动/动作三专家 MoT），推理仍只需一次视觉前向；报告在少量示范和分布偏移下明显优于 Fast-WAM（RoboCasa-GR1、LIBERO、LIBERO-Plus）。
@misc{hu2026trace,
title = {{Trace-WAM: Learning to Act via 3D Motion Prediction for World Action Models}},
author = {Xu Hu and Changjie Wu and Linzhe Shi and Hang Zhang and Chen Yang and Lingjun Zhang and Guowen Zhang and Mu Xu and Junran Peng and Qing Li and Zhaoxiang Zhang},
year = {2026},
eprint = {2610.36840},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36840},
}
@misc{kim2026adapting,
title = {{Adapting Video Models for Robot Control via Motion-guided Asymmetric Prediction}},
author = {Taeyoung Kim and Junyoung Sung and Heeseung Kwon and Minseop Kim and Jinwoo Shin},
year = {2026},
eprint = {2610.36624},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.36624},
}
Shows a fixed world model's uncertainty can break when the deployment policy changes; reweights conformal calibration by a learned state-action ratio. Reports 90% sets covering 55.0% vs 90.9% on real Franka data.指出部署策略一变，固定世界模型的不确定性校准就可能失效；用域分类器学到的状态-动作密度比做加权共形校准，并给出覆盖率保证（部分用 Lean 4 形式化验证）。报告在真实 Franka 轨迹上，普通 90% 共形集合仅覆盖 55.0%，加权后为 90.9%。
@misc{mittal2026calibrated,
title = {{Calibrated for Which Policy? Conformal Uncertainty for Fixed World Models Under Intervention}},
author = {Ayush Mittal and Dhruv Gupta},
year = {2026},
eprint = {2610.35877},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2610.35877},
}
200K+ navigation-planning samples annotated from camera trajectories up to instructions; RL post-training inside a video world model, used as a cheap stand-in environment, reportedly helps on unseen scenes.包含 20 万余条导航规划样本的数据集，场景多样，标注从相机运动轨迹到高层语义指令；并把视频世界模型当作低成本交互环境做强化学习后训练，报告称在未见场景上有明显提升。
@misc{feng2026embodiedworld,
title = {{EmbodiedWorld-200K: Toward Embodied World Agents via Embodied-Planning Dataset and Interactive World Models}},
author = {X. Feng and J. Tang and Z. Lin and L. Chen and S. Shao and Q. Lu and K. Huang},
year = {2026},
eprint = {2610.25959},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2610.25959},
}
Two coupled JEPAs share a latent space: one proposes goal-conditioned latent intentions, the other predicts consequences, closing an intention-action-consequence loop. Evaluated on image-goal navigation in Habitat MP3D.两个耦合的 JEPA 共享隐空间：一个按目标提出隐式“意图”，另一个预测动作后果，形成意图-动作-后果闭环，让世界模型直接参与构造规划；在 Habitat MP3D 图像目标导航上评测。
@misc{zhou2026injepa,
title = {{InJePA: Intention-First Latent Planning with Coupled JEPA World Models for Visual Navigation}},
author = {Siming Zhou and Bowen Ma and Ruiran Du and Pengcheng Zheng and Sheng Zheng and Dongshen Han and Zhijie Li and Wei Dong and Fan Mo and Chaoning Zhang},
year = {2026},
eprint = {2610.23896},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.23896},
}
Dual-system video-action model for VLN: a slow system turns history and instruction into a compact latent plan, a fast one turns plan + latest view into local trajectories; no video is generated at inference.面向视觉语言导航的双系统视频-动作模型：慢系统根据历史观测与指令生成紧凑的隐式规划，快系统结合该规划与最新观测输出局部轨迹；推理时不生成未来视频。代码与权重称将在该仓库发布。
@misc{wang2026havana,
title = {{HAVANA: A Hierarchical Video-Action Model for Vision-Language Navigation}},
author = {Dongzhihan Wang and Yi Du and Chen Wang and Xiuxian Li and Liang Xu},
year = {2026},
eprint = {2610.24122},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.24122},
}
RepresentNav fuses several vision foundation models into a policy-learned space and predicts the next latent before acting (JEPA-style); reports 83.8% on VLNVerse fine-grained (+20.05 pts), real-robot tests.RepresentNav 用一个小型 Transformer 融合多个视觉基础模型各层特征，构建由导航目标自行塑造的表示空间，并在每步动作前预测下一时刻的潜在表示（JEPA 式）；报告 VLNVerse 细粒度成功率 83.8%（领先 20.05 个百分点），并在真机上验证。
@misc{chen2026see,
title = {{See What I Need, Dream What I Want: A Navigation Foundation Model on a Self-Constructed Representation Space}},
author = {Jiawei Chen and Yinxi Wang and Zihan Yang and Meijun Gao and Yaowei Liu and Shuming Zhang and Zecheng Li and Xiangju Mi and Yuxuan Liu and Runzhong Huang and Tianlin Li and Xun Zhang and Zhengyang Wang and Xu Miao and Tong Yang},
year = {2026},
eprint = {2610.31762},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31762},
}
Benchmark of 800 human-rated robot/hand videos from 10 generators to test MLLM judges on physical adherence and instruction alignment, plus a 12K-video training set and RoboJudge-9B (Pearson 0.719 with humans).评测多模态大模型能否评判具身视频生成：800 条人工标注视频（10 个生成模型，夹爪/人手/灵巧手），考察物理合理性与指令一致性；另提供 1.2 万条训练视频及 RoboJudge-9B（与人类评分 Pearson 0.719）。
@misc{ma2026robojudge,
title = {{RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation}},
author = {Siyuan Ma and Yu Qi and Cheng Ye and Ruotian Peng and Xinyi Xu and Zitiantao Lin and Zedong Cai and Yijian Huang and Weiyang Liu and Siyuan Wang and Yandong Wen},
year = {2026},
eprint = {2610.27782},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.27782},
}
@misc{nvidia2025cosmos,
title = {{Cosmos World Foundation Model Platform for Physical AI}},
author = {NVIDIA and : and Niket Agarwal and Arslan Ali and Maciej Bala and Yogesh Balaji and Erik Barker and Tiffany Cai and Prithvijit Chattopadhyay and Yongxin Chen and Yin Cui and Yifan Ding and Daniel Dworakowski and Jiaojiao Fan and Michele Fenzi and Francesco Ferroni and Sanja Fidler and Dieter Fox and Songwei Ge and Yunhao Ge and Jinwei Gu and Siddharth Gururani and Ethan He and Jiahui Huang and Jacob Huffman and Pooya Jannaty and Jingyi Jin and Seung Wook Kim and Gergely Klár and Grace Lam and Shiyi Lan and Laura Leal-Taixe and Anqi Li and Zhaoshuo Li and Chen-Hsuan Lin and Tsung-Yi Lin and Huan Ling and Ming-Yu Liu and Xian Liu and Alice Luo and Qianli Ma and Hanzi Mao and Kaichun Mo and Arsalan Mousavian and Seungjun Nah and Sriharsha Niverty and David Page and Despoina Paschalidou and Zeeshan Patel and Lindsey Pavao and others},
year = {2025},
eprint = {2501.03575},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2501.03575},
}
A self-supervised video encoder, post-trained with a little robot data, plans zero-shot pick-and-place by latent prediction.自监督视频编码器经少量机器人数据后训练，通过潜空间预测实现零样本抓放规划。
@misc{assran2025v,
title = {{V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning}},
author = {Mido Assran and Adrien Bardes and David Fan and Quentin Garrido and Russell Howes and Mojtaba and Komeili and Matthew Muckley and Ammar Rizvi and Claire Roberts and Koustuv Sinha and Artem Zholus and Sergio Arnaud and Abha Gejji and Ada Martin and Francois Robert Hogan and Daniel Dugas and Piotr Bojanowski and Vasil Khalidov and Patrick Labatut and Francisco Massa and Marc Szafraniec and Kapil Krishnakumar and Yong Li and Xiaodong Ma and Sarath Chandar and Franziska Meier and Yann LeCun and Michael Rabbat and Nicolas Ballas},
year = {2025},
eprint = {2506.09985},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2506.09985},
}
VPP — fine-tunes Stable Video Diffusion on robot and human videos and feeds its one-pass predictive features to a diffusion policy; reports 4.33 average length on CALVIN ABC→D and real Franka / dexterous-hand results.VPP：在机器人与人类操作视频上微调 Stable Video Diffusion，只做一次前向提取"预测性"视觉特征再交给扩散策略；论文报告 CALVIN ABC→D 平均完成长度 4.33，并有 Franka 与灵巧手真机实验。
@misc{hu2024video,
title = {{Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations}},
author = {Yucheng Hu and Yanjiang Guo and Pengchao Wang and Xiaoyu Chen and Yen-Jen Wang and Jianke Zhang and Koushil Sreenath and Chaochao Lu and Jianyu Chen},
year = {2024},
eprint = {2412.14803},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2412.14803},
}
UVA — one shared latent for video and actions with two light diffusion heads; masked training lets the same model act as policy, video generator and forward/inverse dynamics model, while keeping policy inference fast.UVA：视频与动作共享一个潜表示，配两个轻量扩散头解码；通过掩码训练，同一模型既能当策略，也能做视频生成和正/逆动力学，且策略推理仍然较快。
One autoregressive model that is both a VLA policy and an image-predicting world model; the two help each other on LIBERO, and an attention mask hiding earlier actions curbs error build-up in action chunks.单个自回归模型同时充当 VLA 策略与预测未来图像的世界模型，二者在 LIBERO 上相互促进；论文还发现自回归生成动作块会累积误差，用屏蔽先前动作的注意力掩码可以缓解。
@misc{cen2025worldvla,
title = {{WorldVLA: Towards Autoregressive Action World Model}},
author = {Jun Cen and Chaohui Yu and Hangjie Yuan and Yuming Jiang and Siteng Huang and Jiayan Guo and Xin Li and Yibing Song and Hao Luo and Fan Wang and Deli Zhao and Hao Chen},
year = {2025},
eprint = {2506.21539},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2506.21539},
}
Genie Envisioner (AgiBot) — a video-diffusion world model (GE-Base) plus a flow-matching action decoder (GE-Act), an action-conditioned neural simulator (GE-Sim) and the EWMBench benchmark in one platform.智元 Genie Envisioner：一个平台集成视频扩散世界模型 GE-Base、流匹配动作解码器 GE-Act、动作条件神经仿真器 GE-Sim，以及世界模型评测基准 EWMBench。
@misc{liao2025genie,
title = {{Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation}},
author = {Yue Liao and Pengfei Zhou and Siyuan Huang and Donglin Yang and Shengcong Chen and Yuxin Jiang and Yue Hu and Jingbin Cai and Si Liu and Jianlan Luo and Liliang Chen and Shuicheng Yan and Maoqing Yao and Guanghui Ren},
year = {2025},
eprint = {2508.05635},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2508.05635},
}
Fine-tunes video world models on a robot to dream new demos, labels them with latent actions or an inverse dynamics model, and trains policies on them; a humanoid learns 20+ new behaviors from one-scene teleop data.把视频世界模型微调到目标机器人上“做梦”生成新示范，再用隐动作或逆动力学模型补上动作标签用于训练策略；人形机器人仅凭单一场景的遥操作数据学会 20 多种新行为。
@misc{jang2025dreamgen,
title = {{DreamGen: Unlocking Generalization in Robot Learning through Video World Models}},
author = {Joel Jang and Seonghyeon Ye and Zongyu Lin and Jiannan Xiang and Johan Bjorck and Yu Fang and Fengyuan Hu and Spencer Huang and Kaushil Kundalia and Yen-Chen Lin and Loic Magne and Ajay Mandlekar and Avnish Narayan and You Liang Tan and Guanzhi Wang and Jing Wang and Qi Wang and Yinzhen Xu and Xiaohui Zeng and Kaiyuan Zheng and Ruijie Zheng and Ming-Yu Liu and Luke Zettlemoyer and Dieter Fox and Jan Kautz and Scott Reed and Yuke Zhu and Linxi Fan},
year = {2025},
eprint = {2505.12705},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2505.12705},
}
A 3D-aware LLM with scene, object and action tokens, aligned with diffusion models so it can imagine goal images and point clouds before planning actions; trained on ~2M 3D-language-action pairs the authors assembled.具备三维感知的大语言模型，引入场景、物体与动作交互 token，并与扩散模型对齐，可在规划动作前“想象”目标图像和点云；训练数据为作者整理的约 200 万条三维-语言-动作样本。
@misc{zhen2024d,
title = {{3D-VLA: A 3D Vision-Language-Action Generative World Model}},
author = {Haoyu Zhen and Xiaowen Qiu and Peihao Chen and Jincheng Yang and Xin Yan and Yilun Du and Yining Hong and Chuang Gan},
year = {2024},
eprint = {2403.09631},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2403.09631},
}
Pretrained to generate future video on 38M clips, then fine-tuned to predict frames and actions together; reports 97.7% average success over 100+ real tabletop tasks and roughly double GR-1's success in unseen scenes.先在 3800 万段视频上预训练未来帧生成，再在机器人数据上微调，同时预测视频帧与动作轨迹；报告在 100 多个真机桌面任务上平均成功率 97.7%，在未见场景中成功率约为 GR-1 的两倍。
@misc{cheang2024gr,
title = {{GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation}},
author = {Chi-Lam Cheang and Guangzeng Chen and Ya Jing and Tao Kong and Hang Li and Yifeng Li and Yuxiao Liu and Hongtao Wu and Jiafeng Xu and Yichu Yang and Hanbo Zhang and Minzhao Zhu},
year = {2024},
eprint = {2410.06158},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.06158},
}
Seer forecasts the next visual state and feeds it to an inverse dynamics model that outputs actions, trained end to end and pretrained on DROID; reports 4.28 avg. length on CALVIN ABC-D plus LIBERO and real-robot gains.Seer 先预测未来视觉状态，再由逆动力学模型据此输出动作，端到端训练并在 DROID 上预训练；报告 CALVIN ABC-D 平均任务长度 4.28，在 LIBERO-LONG 和真机任务上也有提升。
@misc{tian2024predictive,
title = {{Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation}},
author = {Yang Tian and Sizhe Yang and Jia Zeng and Ping Wang and Dahua Lin and Hao Dong and Jiangmiao Pang},
year = {2024},
eprint = {2412.15109},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2412.15109},
}
@misc{bruce2024genie,
title = {{Genie: Generative Interactive Environments}},
author = {Jake Bruce and Michael Dennis and Ashley Edwards and Jack Parker-Holder and Yuge Shi and Edward Hughes and Matthew Lai and Aditi Mavalankar and Richie Steigerwald and Chris Apps and Yusuf Aytar and Sarah Bechtle and Feryal Behbahani and Stephanie Chan and Nicolas Heess and Lucy Gonzalez and Simon Osindero and Sherjil Ozair and Scott Reed and Jingwei Zhang and Konrad Zolna and Jeff Clune and Nando de Freitas and Satinder Singh and Tim Rocktäschel},
year = {2024},
eprint = {2402.15391},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2402.15391},
}
Predicts future DINOv2 patch features instead of pixels, so a world model trained on offline data can plan to new visual goals at test time.预测未来的 DINOv2 图像块特征而非像素，使基于离线数据训练的世界模型能在测试时规划到新的视觉目标。
@misc{zhou2024dino,
title = {{DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning}},
author = {Gaoyue Zhou and Hengkai Pan and Yann LeCun and Lerrel Pinto},
year = {2024},
eprint = {2411.04983},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2411.04983},
}
GR-1 — a GPT-style model that predicts both actions and future frames; pre-training on Ego4D video prediction before robot fine-tuning lifts CALVIN unseen-scene success from 53.3% to 85.4% (as reported).GR-1：GPT 式模型同时预测动作与未来画面；先在 Ego4D 上做视频预测预训练、再用机器人数据微调，论文报告 CALVIN 未见场景成功率从 53.3% 提升到 85.4%。
@misc{wu2023unleashing,
title = {{Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation}},
author = {Hongtao Wu and Ya Jing and Chilam Cheang and Guangzeng Chen and Jiafeng Xu and Xinghang Li and Minghuan Liu and Hang Li and Tao Kong},
year = {2023},
eprint = {2312.13139},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2312.13139},
}
UniSim — an action-conditioned video model used as a simulator to train policies that transfer to the real world.UniSim：以动作为条件的视频生成模型充当仿真器，训练出的策略可迁移到真实世界。
@misc{yang2023learning,
title = {{Learning Interactive Real-World Simulators}},
author = {Sherry Yang and Yilun Du and Kamyar Ghasemipour and Jonathan Tompson and Leslie Kaelbling and Dale Schuurmans and Pieter Abbeel},
year = {2023},
eprint = {2310.06114},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2310.06114},
}
An implicit (decoder-free) world model planned with MPC; one set of hyperparameters works across many continuous-control tasks and scales to a multitask agent.无解码器的隐式世界模型配合 MPC 规划；同一组超参数适用于大量连续控制任务，并可扩展为多任务智能体。
IRIS — a discrete autoencoder turns frames into tokens and a Transformer predicts the next tokens; the agent learns in imagination with only ~2 hours of Atari gameplay.IRIS：离散自编码器把画面变成 token，Transformer 预测后续 token；智能体只用约 2 小时的 Atari 游戏经验便在想象中学会策略。
@misc{micheli2022transformers,
title = {{Transformers are Sample-Efficient World Models}},
author = {Vincent Micheli and Eloi Alonso and François Fleuret},
year = {2022},
eprint = {2209.00588},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2209.00588},
}
UniPi — casts decision making as text-conditioned video generation, then recovers actions from the generated frames with an inverse dynamics model.UniPi：把决策问题转化为文本条件的视频生成，再用逆动力学模型从生成的画面中恢复动作。
@misc{du2023learning,
title = {{Learning Universal Policies via Text-Guided Video Generation}},
author = {Yilun Du and Mengjiao Yang and Bo Dai and Hanjun Dai and Ofir Nachum and Joshua B. Tenenbaum and Dale Schuurmans and Pieter Abbeel},
year = {2023},
eprint = {2302.00111},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2302.00111},
}
@misc{wu2022daydreamer,
title = {{DayDreamer: World Models for Physical Robot Learning}},
author = {Philipp Wu and Alejandro Escontrela and Danijar Hafner and Ken Goldberg and Pieter Abbeel},
year = {2022},
eprint = {2206.14176},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2206.14176},
}
World Models世界模型Quadruped / legged四足 / 足式realquadrupedarmmodel-based-rl
TD-MPC — a task-oriented latent model used for short-horizon planning, with a learned value function covering the long-term return; the predecessor of TD-MPC2.TD-MPC：用面向任务的潜空间模型做短视距规划，由学习到的价值函数估计长期回报；是 TD-MPC2 的前身。
Diffuser — treats planning as denoising whole trajectories, so goals and rewards can steer sampling; an early link between diffusion and control.Diffuser：把规划视为对整条轨迹去噪，目标与奖励可以引导采样；扩散模型与控制结合的早期代表作。
@misc{janner2022planning,
title = {{Planning with Diffusion for Flexible Behavior Synthesis}},
author = {Michael Janner and Yilun Du and Joshua B. Tenenbaum and Sergey Levine},
year = {2022},
eprint = {2205.09991},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2205.09991},
}
@misc{hafner2020mastering,
title = {{Mastering Atari with Discrete World Models}},
author = {Danijar Hafner and Timothy Lillicrap and Mohammad Norouzi and Jimmy Ba},
year = {2020},
eprint = {2010.02193},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2010.02193},
}
The first Dreamer — learns a latent world model from pixels and trains the policy entirely on imagined rollouts.第一代 Dreamer：从像素学习潜空间世界模型，并完全在想象的轨迹上训练策略。
@misc{hafner2019dream,
title = {{Dream to Control: Learning Behaviors by Latent Imagination}},
author = {Danijar Hafner and Timothy Lillicrap and Jimmy Ba and Mohammad Norouzi},
year = {2019},
eprint = {1912.01603},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/1912.01603},
}
PlaNet — learns a latent dynamics model (the RSSM later reused by Dreamer) from images and plans actions directly in latent space with CEM.PlaNet：从图像中学习潜空间动力学模型（即后来 Dreamer 沿用的 RSSM），并用 CEM 直接在潜空间中规划动作。
@misc{hafner2018learning,
title = {{Learning Latent Dynamics for Planning from Pixels}},
author = {Danijar Hafner and Timothy Lillicrap and Ian Fischer and Ruben Villegas and David Ha and Honglak Lee and James Davidson},
year = {2018},
eprint = {1811.04551},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/1811.04551},
}
The classic VAE + RNN agent that learns a compressed model of its environment and trains a controller inside the "dream".经典的 VAE + RNN 智能体，学习环境的压缩模型并在"梦境"中训练控制器。
Guoning Wu (World Agents), Zijian Cai (World Agents), Yuhang Zhang (World Agents), Ying Liu (World Agents) et al.
Abstract摘要
Multi-robot control requires integrating distributed camera views while predicting separate action sequences. Most vision-language-action and world-action models target one robot. Combining multiple views alone does not establish spatial correspondence between them. World Dreamer v5-lite is a multi-robot world-action model built around a shared world state for robot-specific action prediction. Calibrated features from a pretrained video encoder populate this state in workspace coordinates; its spatial size remains fixed as views are added. A vision-language model provides scene context and …
@misc{agents2026world,
title = {{World Dreamer v5-lite: A Multi-Robot World Action Model with One Dreamer}},
author = {Guoning Wu (World Agents) and Zijian Cai (World Agents) and Yuhang Zhang (World Agents) and Ying Liu (World Agents) and Yongbin Zheng (World Agents)},
year = {2026},
eprint = {2610.37518},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37518},
}
Yonghao Yu, Lang Huang, Runyi Li, Toshihiko Yamasaki
Abstract摘要
The vision-language-action (VLA) and world-action model (WAM) communities debate whether control policies should inherit pretrained knowledge from vision-language models or from visual generators. The debate, however, bundles three separable choices: backbone initialization, the objective applied to future observations, and whether the machinery for that future loss is kept at inference. A gap between a VLA and a WAM therefore cannot simply be attributed to any one of them. We show that decoupling these choices yields a common framework for both families, which we term Just Action Model …
@misc{yu2026vision,
title = {{Vision-Language-Action or World-Action? Just Action Models}},
author = {Yonghao Yu and Lang Huang and Runyi Li and Toshihiko Yamasaki},
year = {2026},
eprint = {2610.37219},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37219},
}
World action models (WAMs) predict the visual consequences of the actions they generate, which makes test-time selection a natural way to improve a fixed policy: imagine the futures of candidate actions and execute the best one. The benefit of selection is bounded by the selectable headroom, how much better the best candidate truly is, but the standard estimate, which executes each candidate once and keeps the best outcome, conflates candidate quality with the randomness of everything that follows. We introduce a denoised counterfactual protocol, based on bitwise-reproducible branching, …
@misc{gao2026how,
title = {{How Much Can Best-of-N Help a World Action Model? Measuring Selectable Headroom and Realizing It in Closed Loop}},
author = {Haojia Gao and Yue Cao and Yumin Liu and Haohua Que and Mingkai Liu and Guanqiu Guo and Zhuo Zou and Bokui Chen and others},
year = {2026},
eprint = {2610.37196},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37196},
}
Yang Liu, Xiaoli Liu, Liuzhao Li, Yuduo Song et al.
Abstract摘要
World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency. In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. GAWM compresses the features of each camera view into a small set of tokens. A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at …
@misc{liu2026gawm,
title = {{GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation}},
author = {Yang Liu and Xiaoli Liu and Liuzhao Li and Yuduo Song and Muyuan Liu and Xiang Gao},
year = {2026},
eprint = {2610.37091},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37091},
}
World Action Models (WAMs) learn representations for robot control by coupling future visual prediction with action generation. Successful manipulation depends primarily on how the robot and task-relevant objects evolve, whereas future-video prediction must also reconstruct the appearance and background of the entire scene. We therefore propose Trace-WAM, which augments video-action learning with future 3D motion prediction as an auxiliary objective. This objective supervises temporal point correspondence together with image-plane displacement and depth change, directly describing the …
@misc{hu2026trace,
title = {{Trace-WAM: Learning to Act via 3D Motion Prediction for World Action Models}},
author = {Xu Hu and Changjie Wu and Linzhe Shi and Hang Zhang and Chen Yang and Lingjun Zhang and Guowen Zhang and Mu Xu and others},
year = {2026},
eprint = {2610.36840},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36840},
}
Haoyu Zhao, Yinzhou Tang, Zixuan Wang, Minghao Zhu et al.
Abstract摘要
Learning controllable world dynamics from heterogeneous interaction videos is challenging because robot embodiments use incompatible action spaces, while human videos lack native control labels. We present RynnWorld-Latent, a cross-embodiment video world model that uses the 3D-grounded latent actions introduced by RynnLAM as a shared conditioning interface. This representation enables joint pretraining on human interaction videos, robot demonstrations, and simulation rollouts without requiring a common native action space. Built on a pretrained video transformer, RynnWorld-Latent combines …
@misc{zhao2026rynnworld,
title = {{RynnWorld-Latent: A Cross-Embodiment World Model Conditioned on Universal Latent Actions}},
author = {Haoyu Zhao and Yinzhou Tang and Zixuan Wang and Minghao Zhu and Dongchi Huang and Xingyue Zhao and Qin Zhao and Kehan Li and others},
year = {2026},
eprint = {2610.36791},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36791},
}
Joint-embedding predictive architectures (JEPAs) enable world models to plan from images by predicting future embeddings. However, their visual prediction objectives do not explicitly incorporate task-relevant state information, so the physical state that determines task progress may not be reliably captured in the latent space. Supervising world models with simulator state can make planning more informative, but it requires deciding which state features matter. Features are typically selected by hand or weighted uniformly, and a uniform weighting gives every standardized state feature equal …
@misc{hu2026swim,
title = {{SWIM: Learning What Matters from Behaviors for JEPA World Models}},
author = {Jiaming Hu and Shi Bo and Yan Zheng and Tian Wang and Florian Dubost and Alejandro Mottini and Junze Liu and Arvind Srinivasan and others},
year = {2026},
eprint = {2610.36662},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36662},
}
Taeyoung Kim, Junyoung Sung, Heeseung Kwon, Minseop Kim et al.
Abstract摘要
World Action Models (WAMs) leverage pretrained video generative models for robotic control by jointly predicting future visual observations and actions. The visual prediction objective plays a key role in adapting pretrained video representations to the target robot domain. Yet standard visual prediction inherited from video pretraining treats all visual regions uniformly, despite their differing relevance to robotic control. We therefore investigate the spatial allocation of visual supervision in WAMs and show that focusing supervision on dynamic regions can substantially improve policy …
@misc{kim2026adapting,
title = {{Adapting Video Models for Robot Control via Motion-guided Asymmetric Prediction}},
author = {Taeyoung Kim and Junyoung Sung and Heeseung Kwon and Minseop Kim and Jinwoo Shin},
year = {2026},
eprint = {2610.36624},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.36624},
}
Jiayi Luo, Yang Zhang, Liliang Chen, Nan Wang et al.
Abstract摘要
Action-conditioned world models offer a scalable alternative to costly real-world interaction for improving robot policies. However, existing world-model-based methods largely treat world models as forward-only surrogates of the physical environment, leaving untapped a capability no physical environment affords: rewinding to past decision points to explore counterfactual futures. We introduce Rollback, a policy improvement paradigm that turns world models from passive simulators into counterfactual engines: it rolls back to critical decision points, branches into alternative actions, and …
@misc{luo2026rollback,
title = {{Rollback: Experience Backtracking in World Models for Policy Improvement}},
author = {Jiayi Luo and Yang Zhang and Liliang Chen and Nan Wang and Chen Gao and Boxiang Qiu and Zhuoxuan Li and Baorui Peng and others},
year = {2026},
eprint = {2610.36068},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36068},
}
Era AI Lab Team, Jianhao Lai, Zhihao Zhan, Zhiquan Liao et al.
Abstract摘要
World-action models (WAMs) offer a promising framework for embodied intelligence, yet mainstream approaches remain centered on video foundation models or vision-language models. Even when geometric features extracted by 3D models such as VGGT are incorporated, they are typically passed into a video model or VLM that serves as the primary backbone, leaving 3D geometry as an auxiliary input or prediction target rather than the native foundation for world modeling. We present E0.5, a geometry-native world-action model built directly on 3D foundation models. E0.5 introduces two key innovations. …
@misc{team2026e,
title = {{E0.5: From Reconstructive to Future-Predictive 3D Foundations for Robotic World-Action Modeling}},
author = {Era AI Lab Team and Jianhao Lai and Zhihao Zhan and Zhiquan Liao and Qichang Li and Shubin Chen and Zhiyuan Liu and Hongsen Lei and others},
year = {2026},
eprint = {2610.36011},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36011},
}
Changing a deployment policy can invalidate a world model's uncertainty while leaving the physical transition mechanism and predictor unchanged. For one-step prediction, if source and target laws share the conditional mechanism $P(Y|X)=Q(Y|X)$, then the intervention is identified by the pre-outcome change of measure $w(X)=dQ_X/dP_X$. For policy-controlled dynamics this ratio factors into state-occupancy and action-propensity terms. Weighted conformal calibration transports marginal coverage to the target law; with an estimated ratio $\hat w$, $P_Q\{Y \in C_{\hat w}(X)\} \ge 1-\alpha-TV(Q,\hat …
@misc{mittal2026calibrated,
title = {{Calibrated for Which Policy? Conformal Uncertainty for Fixed World Models Under Intervention}},
author = {Ayush Mittal and Dhruv Gupta},
year = {2026},
eprint = {2610.35877},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35877},
}
Guowen Zhang, Liyi Chen, Xu Hu, Pengfei Wang et al.
Abstract摘要
Direct-policy World Action Models (WAM) have demonstrated the benefits of decoupling action generation from explicit future visual prediction for efficient inference in autonomous driving and embodied control. However, they still require tens of denoising steps to achieve competitive action quality, incurring substantial latency and hindering real-time deployment. Simply reducing the number of denoising steps will collapse the nonlinear multi-step denoising trajectory into coarse updates, substantially degrading action quality. To reduce the denoising steps while enabling real-time action …
@misc{zhang2026oswam,
title = {{OSWAM: An Effective World Action Model with One-Step Action Generation}},
author = {Guowen Zhang and Liyi Chen and Xu Hu and Pengfei Wang and Jingfan Chen and Chenhang He and Lei Zhang},
year = {2026},
eprint = {2610.35850},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35850},
}
Frontier multimodal large language models (MLLMs) can control robots without robot-specific training, but how their capabilities compare and interact with post-trained vision-language-action (VLA) and world action model (WAM) policies across manipulation demands remains to be thoroughly evaluated. In this report, we evaluate GPT-6-Astra on EBench as the core of an agent that controls a robot through an agent harness under the same visual and proprioceptive observation protocol as post-trained policies. We study two agent configurations: direct control, in which the agent generates waypoint …
@misc{yao2026profiling,
title = {{Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy}},
author = {Yifei Yao and Yude Zou and Guanbo Jiao and Xing Gao and Ning Gao and Yukai Wang and Jiantong Chen and Zihou Zhu and others},
year = {2026},
eprint = {2610.35832},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35832},
}
Latent world models are typically pre-trained offline and kept frozen at test time, so planning with them can degrade under distribution shift. Test-time adaptation (TTA) can mitigate this degradation by updating the model during deployment. Unlike pre-training on fixed data, TTA for planning is closed-loop: the planner uses model rollouts to select actions, and the resulting transitions drive the next update. This raises the question: which objective should guide TTA for planning? Under a prediction-only objective, the model is fit only to the outcomes of its own actions, so it can remain …
@misc{mete2026adapt,
title = {{Adapt for Planning, Not just Prediction: Optimistic Test-Time World Model Adaptation}},
author = {Akshay Mete and Dileep Kalathil and P. R. Kumar},
year = {2026},
eprint = {2610.35636},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35636},
}
Jingbo Xu, Hanchao Leng, JiaHui Huang, Hanbing Li et al.
Abstract摘要
World models are emerging as a fundamental component of policy learning for real-world agents, enabling agents to predict future world states and plan actions within a learned representation of the dynamic environment. Existing approaches, however, often model appearance, metric geometry represented by projected LiDAR depth, and ego motion either partially or independently, overlooking the fact that these modalities are different projections of a shared underlying world state. This separation can lead to information loss and cross-modal inconsistency, ultimately limiting downstream …
@misc{xu2026driveunitok,
title = {{DriveUniTok: A Unified Multimodal Tokenizer for Driving World-Action Models}},
author = {Jingbo Xu and Hanchao Leng and JiaHui Huang and Hanbing Li and Jingwei Zhao and Kuiyuan Yang and Chen Gao and Guang Chen and others},
year = {2026},
eprint = {2610.35044},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35044},
}
Video world models trained on large-scale robot data promise a safe and scalable approach for learning and evaluating policies that act in the physical world. However, they still struggle with long-horizon rollouts and out-of-distribution generalization, and they remain computationally demanding at inference time. Typically, video world models make predictions in the latent space of a reconstruction-trained autoencoder, which however is not directly optimized for dynamics modeling. On the other hand, world models designed for latent planning usually optimize latents to be predictable, but …
Jonathan DeCastro, Thomas Balch, Emily Sumner, Deepak Gopinath et al.
Abstract摘要
Assistive agents in shared-control settings face a dual challenge: acting proactively to align with the human's intentions while remaining legible enough for the human to comprehend and fluently respond in real time. We present Dream2Assist, a model-based reinforcement learning framework for tightly-coupled shared decision making that addresses both sides of this challenge. Our approach extends a recurrent state-space world model with intent inference, enabling an assistive policy trained via fictitious co-play to anticipate and support discrete human objectives without overriding human …
@misc{decastro2026dream,
title = {{Dream to Assist: Intent-Conditioned World Models for Shared Vehicle Control}},
author = {Jonathan DeCastro and Thomas Balch and Emily Sumner and Deepak Gopinath and Laporsha Dees and Patricio Reyes Gomez and Elliot Weiss and Megan Applegate-Kenton and others},
year = {2026},
eprint = {2610.34650},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34650},
}
Lingjun Mao, Jinglin Cao, Lukun He, Wenpeng Xu et al.
Abstract摘要
Recent advances in vision-language-action (VLA) and world-action models (WAMs) have significantly improved robotic manipulation, yet deploying robots in the open world demands a higher level of autonomy beyond executing predefined tasks. Robot agents offer a promising paradigm for this transition, integrating task discovery, contextual reasoning, physical execution, and adaptive recovery into a closed interaction loop. However, existing benchmarks largely evaluate isolated capabilities, failing to capture the complexity of open-world robot autonomy. We introduce RobotAgentBench (RAB), a …
@misc{mao2026robotagentbench,
title = {{RobotAgentBench: Towards Open-world Robot Autonomy}},
author = {Lingjun Mao and Jinglin Cao and Lukun He and Wenpeng Xu and Yuchen Yan and Hengyu Jin and Zijun Zhang and Jinzhou Tang and others},
year = {2026},
eprint = {2610.34649},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34649},
}
Hanwen Cui, Yuanyuan Li, Qing Guo, Zhida Zhang et al.
Abstract摘要
World Action Models (WAMs) combine future-video prediction with robot action learning, yet improvements in task success alone do not reveal how predictive supervision manifests in execution behavior or whether its benefits persist under task and environmental shifts. We conduct controlled future-video prediction loss ablations within FastWAM and OpenWAM, keeping model architecture, initialization, robot training data, and training steps fixed within each model family. Under clean conditions on seen tasks, predictive supervision increases success rates by 24.38 and 72.91 percentage points, …
@misc{cui2026what,
title = {{What Does World Modeling Actually Improve in Robot Action Learning?}},
author = {Hanwen Cui and Yuanyuan Li and Qing Guo and Zhida Zhang and Chao Chen and Xi Chen and Jie Cao},
year = {2026},
eprint = {2610.34593},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34593},
}
Chao Chen, Zhenrong Wang, Jilong Wang, Yusen Feng et al.
Abstract摘要
Humanoid robots have recently achieved remarkable success in motion tracking. However, during real-world deployment, robots inevitably encounter dynamics gaps arising from either extra physical disturbances or inaccurate simulation modeling, resulting in a significant training-deployment gap. Therefore, effectively leveraging real-world data to fine-tune policies represents the crucial last mile for robust deployment. To bridge this gap, we propose HumanoidNDM, an online fine-tuning framework for humanoid motion tracking driven by a real-world-trained neural dynamics model. First, in the data …
@misc{chen2026humanoidndm,
title = {{HumanoidNDM: Closing the Training-Deployment Gap of Motion Tracking via Humanoid Neural Dynamics Model}},
author = {Chao Chen and Zhenrong Wang and Jilong Wang and Yusen Feng and Jiangran Lyu and Li Yi and He Wang},
year = {2026},
eprint = {2610.34126},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34126},
}
Doanh Le Thien, Viet-Thanh Nguyen, Pham Tri Quang, Nhi Pham et al.
Abstract摘要
Video world models (VWMs) are rapidly emerging as scalable neural simulators for robotic manipulation. However, most benchmarks only evaluate them as open-world video generators, using Vision-Language Models (VLMs) that lack strict 3D spatial and kinematic reasoning. To bridge this gap, we introduce RoboWorM, a comprehensive, deployment-aware benchmark that leverages known hardware priors such as robot kinematic trees and fixed camera configurations, to systematically examine structural and spatiotemporal fidelity. RoboWorM curates 1,605 episodes across single-arm, bimanual, and humanoid …
@misc{thien2026roboworm,
title = {{RoboWorM: Evaluating Movement Realism, Consistency and Acceleration Affects in Embodied Video World Models}},
author = {Doanh Le Thien and Viet-Thanh Nguyen and Pham Tri Quang and Nhi Pham and Christopher Wewer and Minh-Loi Nguyen and Hoang H. Tran and Tho Le and others},
year = {2026},
eprint = {2610.33807},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.33807},
}
Zheng Zhang, Xiaoyang Tang, Shuaiqi Cheng, Jiakai Huang et al.
Abstract摘要
Robot policy evaluation informs deployment decisions and policy improvement, but physical testing and simulator preparation require substantial effort. World models offer a complementary route through closed-loop interaction in generated environments. Recent benchmarks have demonstrated their potential as policy evaluators through outcome-based evaluation (i.e., assessing task success or failure). However, such evaluation does not establish whether world models faithfully simulate how policies fail. Misrepresenting failures can lead to incorrect diagnoses of policy weaknesses and make …
@misc{zhang2026worldfailbench,
title = {{WorldFailBench: Do World Models Understand Policy Failures?}},
author = {Zheng Zhang and Xiaoyang Tang and Shuaiqi Cheng and Jiakai Huang and Zhongbo Wu and Ziyi Zhang and Yuan Si and Zeyu Liu and others},
year = {2026},
eprint = {2610.33565},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.33565},
}
Zero-shot object navigation requires an embodied agent to choose where to search while retaining evidence from previous observations. HGWM couples an LLM-generated room-anchor prior with persistent spatial-semantic memory. A coarse stage assigns visual-semantic scores to candidate headings on a 0-10 scale; a fine stage measures symbolic compatibility between observed objects, room context, and the goal prior. Phase-dependent fusion combines these scores with an LLM direction recommendation before local action selection. On HM3D v0.1, the reported HGWM configuration achieves 62.6% success rate …
Fanxing Li, Yuxiang Huang, Shengyang Wang, Fangyu Sun et al.
Abstract摘要
Tasks such as quadrotor dynamic-gate racing require agents to anticipate external motion while maintaining precise high-frequency control. World-model-based reinforcement learning provides a natural solution by learning latent dynamics for future imagination, yet it often entangles environmental evolution and the robot's own dynamics in a single learned rollout, where model bias can affect policy optimization efficiency. We propose DiffDreamer, a differentiable reinforcement learning framework that tightly couples a learned world model with differentiable simulation for actor optimization. In …
@misc{li2026diffdreamer,
title = {{DiffDreamer: Coupling World Models with Differentiable Simulation for Agile Quadrotor Control}},
author = {Fanxing Li and Yuxiang Huang and Shengyang Wang and Fangyu Sun and Shuyu Wu and Tianbao Zhang and Dexin Zuo and Yan Yufei and others},
year = {2026},
eprint = {2610.31617},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31617},
}
Qi Lv, Jiayi Lin, Yuxin Jiang, Jianzong Wang et al.
Abstract摘要
Robot world models simulate how environments evolve in response to robot actions. Latent action models enable learning dynamics from videos by encoding visual transitions as proxy actions. However, reconstruction-driven latent actions mix robot motion with object responses and other scene changes, complicating their correspondence with robot controls. We propose SkeLAM, a skeleton-guided latent action model for controllable robot world modeling. SkeLAM decomposes latent actions into a kinematic component grounded by robot skeleton supervision and a residual component retaining complementary …
@misc{lv2026skeleton,
title = {{Skeleton-Grounded Latent Action Model for Robot World Models}},
author = {Qi Lv and Jiayi Lin and Yuxin Jiang and Jianzong Wang and Xiang Deng and Mike Shou Zheng},
year = {2026},
eprint = {2610.31474},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31474},
}
Joint-embedding predictive architectures (JEPAs) learn world models in latent space, enabling efficient planning but leaving their states opaque. We introduce Inside-JEPA, which places relational channels for ordered entity pairs inside the JEPA latent state and shows that an inverse-dynamics loss can ground these channels without relation labels when its head reads them, either alone or as part of the full state. On Something-Something V2, latent prediction alone performs no better than a random relation head, while inverse-dynamics supervision that recovers actions from channel changes …
Jialin Tian, Zhuofan Li, Xinkai Wang, Bokai Lin et al.
Abstract摘要
Robot demonstrations record what the robot saw and what it did. They do not specify how the 3D world changed in between. Scene-level 3D flow describes it by tracking 3D scene points, providing the world representation with 4D geometry. Prior work has used flow to improve robot policies. Flow served as an intermediate prediction that a separate model converts into actions, as a prior injected at inference time, or as an extra output learned together with the action. In all of these designs, flow stays outside the policy, as a separate module or as an extra prediction head. We introduce …
@misc{tian2026uniflowwam,
title = {{UniFlowWAM: A Unified World-Action Model with Visual Semantics and 4D Geometry}},
author = {Jialin Tian and Zhuofan Li and Xinkai Wang and Bokai Lin and Chenyi Wang and Xinyu Zhan and Cewu Lu and Lixin Yang},
year = {2026},
eprint = {2610.30991},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30991},
}
Hengyi Sim, Quentin Rouxel, Dennis Hong, Darwin Caldwell et al.
Abstract摘要
Robust manipulation of soft objects, such as garments, is a critical skill for service and household robots to adopt, but remains challenging due to its high-dimensional state space, and material-dependent deformations. Humans address this challenge by using mental world models that "imagine" the shape and material properties from partial observations, which enables them to predict how actions affect future states. As such, embedding such world models in robots allows for better generalization in manipulating a wide variety of garments in the real world. However, learned world models …
@misc{sim2026context,
title = {{In-Context Learning World Models for Robot Dual-Arm Manipulation of Soft Garment with Partial Observations}},
author = {Hengyi Sim and Quentin Rouxel and Dennis Hong and Darwin Caldwell and Fei Chen},
year = {2026},
eprint = {2610.30988},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30988},
}
Adam Lee, Mutian Xu, Yukang Cao, Shulin Tian et al.
Abstract摘要
Before acting on an object, a robot needs to anticipate how its hand motion will change the physical world and what the resulting scene will look like. Existing approaches typically entangle action-conditioned dynamics, camera motion, and visual synthesis within a video generation model, or represent dynamics as point trajectories without explicit object structure. We introduce DeXplicit, a dexterous manipulation world model that instead predicts how the world evolves explicitly in metric 3D before rendering its visual consequences. Given an initial scene and a future hand trajectory, a …
@misc{lee2026dexplicit,
title = {{DeXplicit: Making Physical State Explicit in Dexterous Manipulation World Models}},
author = {Adam Lee and Mutian Xu and Yukang Cao and Shulin Tian and Ziwei Liu},
year = {2026},
eprint = {2610.30781},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.30781},
}
Huiwon Jang, Suhyeok Jang, Hyeonmo Kang, Seungku Kim et al.
Abstract摘要
World Action Models (WAMs) have emerged as a promising approach to learning generalist robot policies using rich spatiotemporal priors from pre-trained video diffusion models. A recent line of WAMs improves inference efficiency by conditioning a separate action decoder on intermediate future video representations extracted from a single denoising step, avoiding costly iterative denoising required for video generation. This design makes action prediction depend directly on the quality of these future representations. In this paper, we introduce Future REpresentation learning via …
@misc{jang2026self,
title = {{Self-Distilling Future Video Representations for World Action Models}},
author = {Huiwon Jang and Suhyeok Jang and Hyeonmo Kang and Seungku Kim and Jinwoo Shin and Sihyun Yu},
year = {2026},
eprint = {2610.30320},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30320},
}
Zhuoyang Liu, Dantong Niu, Botong Zhang, Boning Shao et al.
Abstract摘要
Recent world action models provide visual foresight and future prediction for robot manipulation, but lack explicit mechanisms for compositional skill reuse. Agentic robotics combines code-based planning and reasoning with reusable skills, but often lacks predictive grounding in physical dynamics. We introduce AW-0, an agentic world action model framework that integrates code-based planning with joint video and action prediction. We jointly model code, future video, and robot actions as three coupled generative streams within a unified mixture-of-transformers architecture. Within this …
@misc{liu2026agentic,
title = {{Agentic World Action Models}},
author = {Zhuoyang Liu and Dantong Niu and Botong Zhang and Boning Shao and Jing Wang and Ruijie Zheng and Yuvan Sharma and Runyu Lu and others},
year = {2026},
eprint = {2610.30136},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30136},
}
Yunhao Luo, Wentao He, Yinpei Dai, Hyelin Nam et al.
Abstract摘要
Existing world models typically perceive a scene through current visual frame or a narrow history window. However, these observations may not reveal the physics properties that determine transition dynamics, such as an object's mass distribution, joint damping, or surface friction. Without calibrating these physics factors for world models, predictions tend to regress toward an empirical prior and induce severe drift in novel test scenes. To this end, we propose PhysiCalWorld, which calibrates action-conditioned world models to the instance-specific dynamics via in-context prompting, without …
@misc{luo2026physicalworld,
title = {{PhysiCalWorld: In-Context Physics Calibration for World Models}},
author = {Yunhao Luo and Wentao He and Yinpei Dai and Hyelin Nam and Hongze Fu and Joyce Chai and Nima Fazeli and Jeong Joon Park},
year = {2026},
eprint = {2610.30125},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30125},
}
Force-sensitive manipulation requires contact and load information that is not provided explicitly by visual observations. We present CW-WAM, a world action model that jointly predicts video, motion, and contact wrench. Inspired by hybrid motion-force control, it contains a unified interaction module that treats both motion and contact wrench as modeling targets and predicts time-aligned motion-wrench chunks as control references. A pair-causal attention mechanism structures the motion-wrench dependencies, while a long fixed-length wrench memory retains past interaction information. The …
@misc{ding2026cw,
title = {{CW-WAM: A Structured Contact-Wrench World Action Model for Force-Sensitive Manipulation}},
author = {Houzhu Ding and Xinda Qi and Luqi Wang and Ming Yang},
year = {2026},
eprint = {2610.30117},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30117},
}
Yuan Xu, Yixiang Chen, Qisen Ma, Jiabing Yang et al.
Abstract摘要
World Action Models (WAMs) have shown strong potential for robot manipulation by jointly learning visual dynamics and executable actions. However, their video and action objectives do not explicitly constrain the 3D geometry that is crucial for precise manipulation. Recently, geometry-aware WAMs introduce depth or 4D prediction branches, whereas representation alignment methods supervise intermediate WAM features with pretrained 3D foundation models. Yet we find that such pointwise alignment benefits the video features but degrades action learning, since dense geometric feature targets overly …
@misc{xu2026relational,
title = {{Relational Geometry Forcing for World Action Models}},
author = {Yuan Xu and Yixiang Chen and Qisen Ma and Jiabing Yang and Peiyan Li and Kai Wang and Jianhua Yang and Jianlou Si and others},
year = {2026},
eprint = {2610.29282},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.29282},
}
World action models (WAMs) are shown to substantially diverge under generalization performance based on their generative backbones. We investigate this gap through the representational geometry that WAMs inherit from their backbones and how policy training reorganizes it. Our central hypothesis is that a useful representational prior would structure geometry that separates nuisance information from the in-distribution manifold in advance of training as a policy. We quantify this with nuisance orthogonality; orthogonality of the feature under visual variation outside the empirical …
World models predict how embodied systems evolve under actions, but their accuracy can degrade when physical conditions move outside the training distribution. We study two components of causal modeling: which inputs a predictor may use, and how its mechanism computes their effects. In a synthetic benchmark with shortcut features, a frozen dependency mask derived from paired interventions reduces mean out-of-distribution (OOD) RMSE from 1.769 for a fully connected predictor to 0.697, close to the generator-oracle mask (0.694). For robot link-mass extrapolation in CausalWorld, with disjoint …
@misc{lin2026causal,
title = {{Causal world models under physical parameter shifts}},
author = {Chu Lin and Yi Sun},
year = {2026},
eprint = {2610.29088},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.29088},
}
Rahul Sajnani, Diego Romeres, Mingyuan Zhang, Kai Gao et al.
Abstract摘要
Motivated by the human ability to integrate experience across language, perception, and action, we introduce UniWALM, a unified world-action-language model that learns video, action, and language in a single shared representation, where each modality is both a conditioning signal and a supervision target. We ask: can jointly generating language enrich this shared representation, improving manipulation and supporting memory-dependent behavior? Unlike prior approaches that attach a reasoning or vision-language model to a video-action backbone and run them as distinct stages, UniWALM unifies all …
@misc{sajnani2026uniwalm,
title = {{UniWALM: Unified World Action Language Model for Embodied Robot Control}},
author = {Rahul Sajnani and Diego Romeres and Mingyuan Zhang and Kai Gao and Carl Qi and Baichuan Huang and Srinath Sridhar and Dylan Randle and others},
year = {2026},
eprint = {2610.28626},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28626},
}
Zebin Xing, Yuhang Zheng, Yupeng Zheng, Yichen Zhang et al.
Abstract摘要
Evaluating and post-training robot policies through physical interaction is costly and difficult to scale, motivating action-conditioned world models as interactive simulators. However, visually plausible predictions can misrepresent contact dynamics, providing unreliable feedback for policy evaluation and learning. We introduce PhysCanvas, a state-consistent visuo-tactile world model for contact-rich robot manipulation. PhysCanvas unifies multi-view RGB, tactile images, marker displacement, task state, and actions within a unified vision space. All modalities share a video VAE and pretrained …
@misc{xing2026physical,
title = {{Physical Canvas: A Vision Tactile World Model For Robot Policy Evaluation And Learning}},
author = {Zebin Xing and Yuhang Zheng and Yupeng Zheng and Yichen Zhang and Zichen Li and Shangqing Zhou and Yujie Zang and Yuchen Liu and others},
year = {2026},
eprint = {2610.28613},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28613},
}
Qiayuan Liao, Yiyang Shao, Junfeng Long, Kevin Zakka et al.
Abstract摘要
Tracking large human-motion datasets has become the dominant route to general, expressive humanoid whole-body control, but scaling across many motions is expensive. Current pipelines use model-free reinforcement learning, either merging per-motion specialists through lossy distillation, or brute-forcing a single policy with cluster-scale compute. SuperMimic amortizes learning across all motions with one differentiable world model of their shared dynamics, fit from logged transitions of a standard rigid-body simulator. The world model and the tracking policy train together in one loop, and …
@misc{liao2026supermimic,
title = {{SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model}},
author = {Qiayuan Liao and Yiyang Shao and Junfeng Long and Kevin Zakka and Roberto Horowitz and Pieter Abbeel and Koushil Sreenath},
year = {2026},
eprint = {2610.28480},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28480},
}
Zhuohao Li, Jiawei He, Yuxuan Yan, Lang Zhou et al.
Abstract摘要
World-Action Models (WAMs) have emerged as a promising paradigm for embodied generalist policies by jointly modeling future world evolution and robot actions across diverse manipulation tasks. However, representing diverse futures is only the first step: a capable WAM must also generate futures that match the requirements of the current task and execution progress. In long-horizon manipulation where similar observation-action segments occur across different tasks and stages, existing approaches may generate locally plausible yet misdirected rollouts to drift toward wrong task stage, i.e., a …
@misc{li2026plausible,
title = {{Plausible Future, Wrong Choice: Steering World-Action Models toward Intended Future}},
author = {Zhuohao Li and Jiawei He and Yuxuan Yan and Lang Zhou and Yinghao Li and Yanrui Li and Tianyu Zhang and Yi-Yang Song and others},
year = {2026},
eprint = {2610.28431},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28431},
}