Thousands of parallel GPU environments train a quadruped walking policy in minutes; legged_gym became a standard starter.数千个 GPU 并行环境几分钟就能训练出四足行走策略；legged_gym 成为标准入门框架。
@misc{rudin2021learning,
title = {{Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning}},
author = {Nikita Rudin and David Hoeller and Philipp Reist and Marco Hutter},
year = {2021},
eprint = {2109.11978},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2109.11978},
}
Asks whether joint readings alone suffice for millimetre-precise insertion — trains sim-to-real policies with no camera, force or tactile input.探究仅靠关节读数能否完成毫米级精密插装：训练不用相机、力或触觉输入的 sim-to-real 策略。
Sim-only RL recipe for in-hand object pose reaching on a LEAP hand, zero-shot to hardware; reports 95.2% of 4,800 unseen goals reached and a tunable residual safety filter that cuts drops by 75%.在仿真中无需人类数据训练 LEAP 灵巧手的手内物体位姿到达策略，零样本迁移到真机；报告在 4800 个未见目标上 95.2% 到达，可调节的残差安全滤波器使掉落减少 75%。
@misc{chuang2026pushing,
title = {{Pushing Dexterity and Safety for In-Hand Object Pose Reaching That Works Remarkably Well}},
author = {Ian Chuang and Hiya Shah and Vamsi Chilamkur and Iman Soltani and Claire Tomlin},
year = {2026},
eprint = {2610.37652},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37652},
}
Real-to-sim-to-real: smartphone scans become physics-ready sim scenes, whole-body motions are corrected by rollouts, giving 4,800 demos; a sim-only GR00T N1.7 on Unitree G1 reports 66.7% vs. 46.7% trained on real data.真实到仿真再到真实：用手机拍摄重建可物理仿真的场景，并根据仿真回放修正全身动作，生成 4800 条示教；仅用仿真数据训练的 GR00T N1.7 在宇树 G1 上报告平均成功率 66.7%，真实数据训练为 46.7%。
@misc{yang2026zero,
title = {{Zero-HLM: Generalizable Humanoid Loco-Manipulation without Real Robot Data}},
author = {Dianyi Yang and Mengya Liu and Zhumei Wang and Yu Liu and Xiaochen Zhang and Yixuan Li and Le Ma and Yuhang Yang and Weiqi Huang and Wei Liang and Baoxiong Jia and Siyuan Huang},
year = {2026},
eprint = {2610.35075},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35075},
}
MetaGenSim learns which simulated data to generate, using meta-gradients from real-world performance, instead of sampling the simulator uniformly.MetaGenSim 利用来自真实世界表现的元梯度，学习应该生成哪些仿真数据，而不是对仿真器做均匀采样。
@misc{ma2026active,
title = {{Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training}},
author = {Liqian Ma and Vaibhav Saxena and Utkarsh Aashu Mishra and Shuo Cheng and Yongxin Chen and Danfei Xu},
year = {2026},
eprint = {2610.19479},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.19479},
}
GPU batch-rendering quadrotor simulator; reports 20×+ the speed of existing batch quadrotor sims, with a new ray tracer rendering RGB, depth, segmentation, normals and optical flow (CoRL 2026).以 GPU 批量渲染为核心的四旋翼仿真器；报告比现有批量四旋翼仿真器快 20 倍以上，新的光线追踪引擎（支持 OptiX、Vulkan、Metal、WebGPU）可渲染 RGB、深度、分割、法向和光流（CoRL 2026）。
@misc{eschmann2026hyperdrone,
title = {{HyperDrone: Simulation-to-Reality Transfer at Millions of Frames per Second}},
author = {Jonas Eschmann and Giuseppe Loianno},
year = {2026},
eprint = {2610.28950},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28950},
}
Generates sim-ready rooms by sampling furniture layouts with diffusion priors, then compiling them into collision-free 3D scenes; nav policies trained there beat ones trained on real scans on held-out real scans.先用扩散先验在功能分区内采样家具布局，再“编译”为无碰撞、可直接仿真的 3D 场景；论文报告在其场景中训练的物体导航策略，在留出的真实扫描场景上优于用等量真实扫描训练的策略。
@misc{liu2026scenebloom,
title = {{SceneBloom: Agentic Indoor Environment Generation with Realistic and Diverse Layouts for Embodied Learning}},
author = {Yuhan Liu and Lianhui Qin and Shenji Wan},
year = {2026},
eprint = {2610.32978},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.32978},
}
@misc{castillaarquillo2026mare,
title = {{MARE: An Open-Source Lunar Rover Simulator with Shadow Ground Truth for Perception and Autonomy}},
author = {Raúl Castilla-Arquillo and Miguel Olivares-Mendez},
year = {2026},
eprint = {2610.28465},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28465},
}
VLA robustness benchmark on RoboCasa365: paired clean/perturbed rollouts differing in one shift, plus stage-level diagnostics of where long-horizon failures start; evaluated on GR00T N1.5.基于 RoboCasa365 的 VLA 鲁棒性诊断基准：成对比较只差一个分布偏移因素的干净/扰动执行，并定位长时序任务中失败从哪个阶段开始，而不只看总成功率；在 GR00T N1.5 上评测。
@misc{wang2026robocasa,
title = {{RoboCasa-Shift: Stage-Aware Robustness Evaluation of Vision-Language-Action Policies in Long-Horizon Household Manipulation}},
author = {Haocheng Wang and Youquan Liu and Weiyu Guo and Aokai Zhang and Donghang Wu and Hongyu Liu and Sha Zhang and Weidong Yang and Ben Fei},
year = {2026},
eprint = {2610.27861},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.27861},
}
Isaac Sim benchmark with 5 arms, 6 dexterous hands, ~200 assets and 18k human demos over 66 tasks; reports the best trained policy (a WAM) at only 29.1% on Core-50, near 0% on precision-fit and deformable tasks.基于 Isaac Sim 的灵巧操作基准：5 种机械臂、6 种灵巧手、约 200 个资产、1.8 万条人类演示、66 个任务；报告最好的训练策略（一种世界动作模型）在 Core-50 上仅 29.1%，精密装配和柔性物体任务接近 0%。
@misc{luo2026handson,
title = {{HandsOn: A Large-Scale Simulation Benchmark for Generalizable Dexterous Manipulation}},
author = {Tianwei Luo and Yiming He and Hongjin Wu and Ting Pan and Debin Ren and Haorui Guo and Junlin Yang and Yuxin Zuo and Zhehao Kou and Kaisen Yang and Zhuoqun Lai and Yi Ding and Jiale Yu and Yuchen Zhang and Zijie Qiu and Haozhan Li and Ning Ding},
year = {2026},
eprint = {2610.26765},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.26765},
}
Benchmark for catching drifting, tumbling objects inside a spacecraft with a fixed-base arm, plus storage and servicing tasks. Reports a frozen generalist policy captured only 2/200 static targets.太空机器人操作基准：用固定基座机械臂抓取舱内漂浮、旋转的物体，另含样本存储与在轨维护任务。报告称所测的冻结通用策略在 200 个静止目标中只抓住 2 个。
@misc{wang2026astrobench,
title = {{AstroBench: Benchmarking Free-Floating Capture and Space Robotic Manipulation}},
author = {Zizhe Wang and Jia Deng and Xin Lu and Zehao Jia and Yi Huang and Yao Zhu},
year = {2026},
eprint = {2610.31064},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31064},
}
BEHAVIOR-Spatial, built on BEHAVIOR-1K: spatial QA, point grounding and robot action tests. A baseline policy drops from 41.6% on held-out layouts to 16.1% when the requested target changes; oracle target cues help most.基于 BEHAVIOR-1K 的 BEHAVIOR-Spatial 基准：包含空间问答、指点定位与机器人动作评测。基线策略在未见布局上成功率 41.6%，但在同一布局中更换目标物体指令时降至 16.1%；提供真值目标提示带来的提升最大。
@misc{chen2026behavior,
title = {{Behavior Spatial Benchmark: Evaluating Spatial Understanding for Robotics}},
author = {Sirui Chen and Huang Huang and Qineng Wang and Wenlong Huang and Manling Li and Ruohan Zhang and Li Fei-Fei and Jiajun Wu},
year = {2026},
eprint = {2610.34051},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34051},
}
Benchmark where a robot must run physics experiments to find an unknown answer: 30 experiments, 5 disciplines, parameters resampled per instance. Reports even the best frontier agent succeeds under half the time.“为求知而行动”的基准：机器人在物理仿真实验室中自行操作仪器、读数，测出每次随机重采样的未知参数。共 30 个实验、5 个学科；报告最强的前沿模型智能体在原生控制下成功率也不到一半，失败多源于与仪器交互。
@misc{gao2026embodiedlab,
title = {{EmbodiedLab: Benchmarking Embodied Agents That Act to Know}},
author = {Hongcheng Gao and Zelin Zheng and Alex Zhou and Jay Zhu and Shijia Ge and Yazhe Wang and Xiping Li and Jianshu Zeng and Di Wu and Xuan Shangguan and Yexing Wan and Lingyu He and Yongli Wang and Xiaohan Yu and Zhiqi Jia and Yue Liu and Bryan Hooi and Yinpeng Dong and Wentao Zhang and Sihang Wu and Xiao He},
year = {2026},
eprint = {2610.35385},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35385},
}
Isaac Lab digital-twin benchmark for language-conditioned spatial tasks on the low-cost SO-101 arm (138 objects, 916 episodes); sim ranks policies exactly as the real arm. Best VLAs ~61%, an LLM-agent harness 83.2%.基于 Isaac Lab 数字孪生的低成本 SO-101 机械臂语言条件空间操作基准（138 个物体、916 个评测回合）；仿真中的策略排名与真机完全一致。最好的 VLA 约 61%，由 LLM 写代码调用抓取策略的方案达 83.2%。
Pretrains a grasping VLA almost only on SynGrasp-1B, a billion-frame photorealistic sim dataset, co-trained with web data; reports zero-shot sim-to-real open-vocabulary grasping and few-shot adaptation.几乎只用十亿帧级的照片级仿真抓取数据集 SynGrasp-1B（并联合网络语义数据）预训练抓取 VLA；论文报告可零样本从仿真迁移到真实开放词汇抓取，并支持少样本适配。
@misc{deng2025graspvla,
title = {{GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data}},
author = {Shengliang Deng and Mi Yan and Songlin Wei and Haixin Ma and Yuxin Yang and Jiayi Chen and Zhiqi Zhang and Taoyu Yang and Xuheng Zhang and Wenhao Zhang and Heming Cui and Zhizheng Zhang and He Wang},
year = {2025},
eprint = {2505.03233},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2505.03233},
}
Genie Envisioner (AgiBot) — a video-diffusion world model (GE-Base) plus a flow-matching action decoder (GE-Act), an action-conditioned neural simulator (GE-Sim) and the EWMBench benchmark in one platform.智元 Genie Envisioner：一个平台集成视频扩散世界模型 GE-Base、流匹配动作解码器 GE-Act、动作条件神经仿真器 GE-Sim，以及世界模型评测基准 EWMBench。
@misc{liao2025genie,
title = {{Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation}},
author = {Yue Liao and Pengfei Zhou and Siyuan Huang and Donglin Yang and Shengcong Chen and Yuxin Jiang and Yue Hu and Jingbin Cai and Si Liu and Jianlan Luo and Liliang Chen and Shuicheng Yan and Maoqing Yao and Guanghui Ren},
year = {2025},
eprint = {2508.05635},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2508.05635},
}
GPU-accelerated MJX environments for locomotion and manipulation that train in minutes and transfer to real robots.基于 MJX 的 GPU 加速运动与操作环境，几分钟即可训练并迁移到真实机器人。
@misc{zakka2025mujoco,
title = {{MuJoCo Playground}},
author = {Kevin Zakka and Baruch Tabanpour and Qiayuan Liao and Mustafa Haiderbhai and Samuel Holt and Jing Yuan Luo and Arthur Allshire and Erik Frey and Koushil Sreenath and Lueder A. Kahrs and Carmelo Sferrazza and Yuval Tassa and Pieter Abbeel},
year = {2025},
eprint = {2502.08844},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2502.08844},
}
Simulation pipeline that writes and checks task code with multimodal LLMs to mass-produce bimanual demos under heavy randomization (clutter, lighting, backgrounds, instructions); also a 50-task dual-arm benchmark.用多模态大模型自动生成并校验任务代码，在强域随机化（杂物、光照、背景、指令）下批量产出双臂演示数据的仿真管线，同时提供 50 个双臂任务的基准。
@misc{chen2025robotwin,
title = {{RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation}},
author = {Tianxing Chen and Zanxin Chen and Baijun Chen and Zijian Cai and Yibin Liu and Zixuan Li and Qiwei Liang and Xianliang Lin and Yiheng Ge and Zhenyu Gu and Weiliang Deng and Yubin Guo and Tian Nian and Xuanbing Xie and Qiangyu Chen and Kailun Su and Tianling Xu and Guodong Liu and Mengkang Hu and Huan-ang Gao and Kaixuan Wang and Zhixuan Liang and Yusen Qin and Xiaokang Yang and Ping Luo and Yao Mu},
year = {2025},
eprint = {2506.18088},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2506.18088},
}
The reference paper for Isaac Lab, Orbit's successor — GPU-parallel physics, rendering and sensor simulation with ready-made robots and tasks for RL and imitation.Isaac Lab（Orbit 的后继）的正式论文：GPU 并行的物理、渲染与传感器仿真，内置机器人与任务，面向强化学习和模仿学习。
@misc{nvidia2025isaac,
title = {{Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning}},
author = {NVIDIA and : and Mayank Mittal and Pascal Roth and James Tigue and Antoine Richard and Octi Zhang and Peter Du and Antonio Serrano-Muñoz and Xinjie Yao and René Zurbrügg and Nikita Rudin and Lukasz Wawrzyniak and Milad Rakhsha and Alain Denzler and Eric Heiden and Ales Borovicka and Ossama Ahmed and Iretiayo Akinola and Abrar Anwar and Mark T. Carlson and Ji Yuan Feng and Animesh Garg and Renato Gasoto and Lionel Gulich and Yijie Guo and M. Gussert and Alex Hansen and Mihir Kulkarni and Chenran Li and Wei Liu and Viktor Makoviychuk and Grzegorz Malczyk and Hammad Mazhar and Masoud Moghani and Adithyavairavan Murali and Michael Noseworthy and Alexander Poddubny and Nathan Ratliff and Welf Rehberg and Clemens Schwarke and Ritvik Singh and James Latham Smith and Bingjie Tang and Ruchik Thaker and Matthew Trepte and Karl Van Wyk and Fangzhou Yu and Alex Millane and Vikram Ramasamy and others},
year = {2025},
eprint = {2511.04831},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2511.04831},
}
Open, GPU-accelerated physics engine built on NVIDIA Warp and OpenUSD with multiple solvers for contact-rich robot learning; co-developed by Disney Research, Google DeepMind and NVIDIA, hosted by the Linux Foundation.基于 NVIDIA Warp 和 OpenUSD 的开源 GPU 物理引擎，支持多种求解器，面向接触丰富的机器人学习；由 Disney Research、Google DeepMind 和 NVIDIA 联合开发，由 Linux 基金会托管。
Isaac Sim platform where an LLM builds tasks and layouts from a task-oriented scene graph over ~10K objects; its 200-scenario benchmark finds modular foundation-model systems generalize better than end-to-end policies.基于 Isaac Sim 的平台：LLM 依据任务导向场景图，从约 1 万个物体中生成任务与布局；其 200 个场景的基准发现，结合基础模型的模块化系统比端到端策略泛化更好。
@misc{gao2025genmanip,
title = {{GenManip: LLM-driven Simulation for Generalizable Instruction-Following Manipulation}},
author = {Ning Gao and Yilun Chen and Shuai Yang and Xinyi Chen and Yang Tian and Hao Li and Haifeng Huang and Hanqing Wang and Tai Wang and Jiangmiao Pang},
year = {2025},
eprint = {2506.10966},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2506.10966},
}
MetaSim puts many simulators (Isaac Sim/Gym, MuJoCo, Genesis, SAPIEN…) behind one interface; adds a large synthetic dataset migrated from public benchmarks and unified IL/RL benchmarks.MetaSim 用统一接口封装多种仿真器（Isaac Sim/Gym、MuJoCo、Genesis、SAPIEN、PyBullet 等）；提供由公开基准迁移而来的大规模合成数据集，以及按泛化层级划分的统一模仿学习/强化学习基准。
@misc{geng2025roboverse,
title = {{RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning}},
author = {Haoran Geng and Feishi Wang and Songlin Wei and Yuyang Li and Bangjun Wang and Boshi An and Charlie Tianyue Cheng and Haozhe Lou and Peihao Li and Yen-Jen Wang and Yutong Liang and Dylan Goetting and Chaoyi Xu and Haozhe Chen and Yuxi Qian and Yiran Geng and Jiageng Mao and Weikang Wan and Mingtong Zhang and Jiangran Lyu and Siheng Zhao and Jiazhao Zhang and Jialiang Zhang and Chengyang Zhao and Haoran Lu and Yufei Ding and Ran Gong and Yuran Wang and Yuxuan Kuang and Ruihai Wu and Baoxiong Jia and Carlo Sferrazza and Hao Dong and Siyuan Huang and Yue Wang and Jitendra Malik and Pieter Abbeel},
year = {2025},
eprint = {2504.18904},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2504.18904},
}
Broad embodied-AI survey — robots, simulators, embodied perception, interaction, agents and sim-to-real — with the role of multimodal and world models; backed by the widely used HCPLab paper list.覆盖面很广的具身智能综述：机器人本体、仿真器、具身感知、交互、具身智能体与 sim-to-real，并讨论多模态大模型与世界模型的作用；配套中山大学 HCPLab 维护的常用论文列表。
@misc{liu2024aligning,
title = {{Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI}},
author = {Yang Liu and Weixing Chen and Yongjie Bai and Xiaodan Liang and Guanbin Li and Wen Gao and Liang Lin},
year = {2024},
eprint = {2407.06886},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2407.06886},
}
GPU-based simulation of GelSight-style tactile images and contact forces in Isaac Gym, plus learning tools for sim-to-real transfer of visuotactile policies.在 Isaac Gym 中用 GPU 模拟 GelSight 式触觉图像与接触力，并提供用于视触觉策略仿真到真实迁移的学习工具。
@misc{akinola2024tacsl,
title = {{TacSL: A Library for Visuotactile Sensor Simulation and Learning}},
author = {Iretiayo Akinola and Jie Xu and Jan Carius and Dieter Fox and Yashraj Narang},
year = {2024},
eprint = {2408.06506},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2408.06506},
}
@misc{tao2024maniskill,
title = {{ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI}},
author = {Stone Tao and Fanbo Xiang and Arth Shukla and Yuzhe Qin and Xander Hinrichsen and Xiaodi Yuan and Chen Bao and Xinsong Lin and Yulin Liu and Tse-kai Chan and Yuan Gao and Xuanlin Li and Tongzhou Mu and Nan Xiao and Arnav Gurha and Viswesh Nagaswamy Rajesh and Yong Woo Choi and Yen-Ru Chen and Zhiao Huang and Roberto Calandra and Rui Chen and Shan Luo and Hao Su},
year = {2024},
eprint = {2410.00425},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.00425},
}
120 kitchen scenes, 2,500+ objects and 100 tasks with generated demonstrations for training generalist policies.120 个厨房场景、2500 多个物体与 100 个任务，并附生成演示，用于训练通用策略。
@misc{nasiriany2024robocasa,
title = {{RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots}},
author = {Soroush Nasiriany and Abhiram Maddukuri and Lance Zhang and Adeet Parikh and Aaron Lo and Abhishek Joshi and Ajay Mandlekar and Yuke Zhu},
year = {2024},
eprint = {2406.02523},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2406.02523},
}
Isaac Gym–based RL framework for humanoid locomotion with a sim-to-sim check in MuJoCo before deploying zero-shot to real humanoids; a common starting point for humanoid RL.基于 Isaac Gym 的人形机器人运动强化学习框架，先在 MuJoCo 中做 sim2sim 验证，再零样本部署到真实人形机器人，是人形 RL 的常用起点。
Extends MimicGen to humanoids with two dexterous hands, turning 60 human demos into 21K simulated ones across bimanual tasks, plus a real-to-sim-to-real pipeline tested on real can sorting.把 MimicGen 扩展到双灵巧手人形机器人，从 60 条人类演示生成 2.1 万条双臂仿真演示，并给出在真实分拣罐子任务上验证的 real-to-sim-to-real 流程。
@misc{jiang2024dexmimicgen,
title = {{DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning}},
author = {Zhenyu Jiang and Yuqi Xie and Kevin Lin and Zhenjia Xu and Weikang Wan and Ajay Mandlekar and Linxi Fan and Yuke Zhu},
year = {2024},
eprint = {2410.24185},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.24185},
}
27 simulated tasks (12 locomotion, 15 whole-body manipulation) on a Unitree H1 with two dexterous hands; reports that DreamerV3, TD-MPC2, SAC and PPO fail most of them, especially long-horizon whole-body ones.27 个仿真任务（12 个运动、15 个全身操作），机器人为带两只灵巧手的 Unitree H1；报告 DreamerV3、TD-MPC2、SAC、PPO 在大多数任务上未达标，长时程全身协调任务尤甚。
@misc{sferrazza2024humanoidbench,
title = {{HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation}},
author = {Carmelo Sferrazza and Dun-Ming Huang and Xingyu Lin and Youngwoon Lee and Pieter Abbeel},
year = {2024},
eprint = {2403.10506},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2403.10506},
}
@misc{mittal2023orbit,
title = {{Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments}},
author = {Mayank Mittal and Calvin Yu and Qinxi Yu and Jingzhou Liu and Nikita Rudin and David Hoeller and Jia Lin Yuan and Ritvik Singh and Yunrong Guo and Hammad Mazhar and Ajay Mandlekar and Buck Babich and Gavriel State and Marco Hutter and Animesh Garg},
year = {2023},
eprint = {2301.04195},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2301.04195},
}
Turns a handful of human demos into thousands by splitting them into object-centric segments and replaying those segments in new scenes; widely reused to scale imitation data.把少量人类演示切分为以物体为中心的片段，并在新场景中变换重放，从而扩增为数千条演示；被广泛用于扩充模仿学习数据。
@misc{mandlekar2023mimicgen,
title = {{MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations}},
author = {Ajay Mandlekar and Soroush Nasiriany and Bowen Wen and Iretiayo Akinola and Yashraj Narang and Linxi Fan and Yuke Zhu and Dieter Fox},
year = {2023},
eprint = {2310.17596},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2310.17596},
}
Adds simulated humanoid avatars and a human-in-the-loop interface to Habitat, so robots can be trained and evaluated on tasks shared with people, such as social navigation and social rearrangement.为 Habitat 加入仿真人形化身和真人在环交互接口，可在与人协作的任务（如社交导航、社交重排）上训练和评测机器人。
@misc{puig2023habitat,
title = {{Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots}},
author = {Xavier Puig and Eric Undersander and Andrew Szot and Mikael Dallaire Cote and Tsung-Yen Yang and Ruslan Partsey and Ruta Desai and Alexander William Clegg and Michal Hlavac and So Yeon Min and Vladimír Vondruš and Theophile Gervet and Vincent-Pierre Berges and John M. Turner and Oleksandr Maksymets and Zsolt Kira and Mrinal Kalakrishnan and Jitendra Malik and Devendra Singh Chaplot and Unnat Jain and Dhruv Batra and Akshara Rai and Roozbeh Mottaghi},
year = {2023},
eprint = {2310.13724},
archivePrefix = {arXiv},
primaryClass = {cs.HC},
url = {https://arxiv.org/abs/2310.13724},
}
Open-source tactile RL environments with sim-to-real transfer across three low-cost optical sensors (TacTip, DIGIT, DigiTac), for comparing them on the same tasks.开源触觉强化学习环境，支持三种低成本光学触觉传感器（TacTip、DIGIT、DigiTac）的仿真到真机迁移，便于在相同任务上比较。
@misc{lin2022tactile,
title = {{Tactile Gym 2.0: Sim-to-real Deep Reinforcement Learning for Comparing Low-cost High-Resolution Robot Touch}},
author = {Yijiong Lin and John Lloyd and Alex Church and Nathan F. Lepora},
year = {2022},
eprint = {2207.10763},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2207.10763},
}
1,000 household activities chosen from a survey of what people want robots to do, defined in a logic language and simulated in OmniGibson with realistic physics and rendering.依据人们希望机器人做什么的调查挑选出 1000 项家务活动，用逻辑语言定义，并在具备真实物理与渲染的 OmniGibson 中仿真。
@misc{li2024behavior,
title = {{BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation}},
author = {Chengshu Li and Ruohan Zhang and Josiah Wong and Cem Gokmen and Sanjana Srivastava and Roberto Martín-Martín and Chen Wang and Gabrael Levine and Wensi Ai and Benjamin Martinez and Hang Yin and Michael Lingelbach and Minjune Hwang and Ayano Hiranaka and Sujay Garlanka and Arman Aydin and Sharon Lee and Jiankai Sun and Mona Anvari and Manasi Sharma and Dhruva Bansal and Samuel Hunter and Kyu-Young Kim and Alan Lou and Caleb R Matthews and Ivan Villa-Renteria and Jerry Huayang Tang and Claire Tang and Fei Xia and Yunzhu Li and Silvio Savarese and Hyowon Gweon and C. Karen Liu and Jiajun Wu and Li Fei-Fei},
year = {2024},
eprint = {2403.09227},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2403.09227},
}
800K+ annotated 3D object models; widely used to populate simulators with diverse objects for grasping, rearrangement and procedural scene generation (Objaverse-XL extends it to 10M+).80 万+ 带标注的 3D 物体模型，常用于为仿真器填充多样物体，用于抓取、重排和程序化场景生成（Objaverse-XL 扩展到 1000 万+）。
@misc{deitke2022objaverse,
title = {{Objaverse: A Universe of Annotated 3D Objects}},
author = {Matt Deitke and Dustin Schwenk and Jordi Salvador and Luca Weihs and Oscar Michel and Eli VanderBilt and Ludwig Schmidt and Kiana Ehsani and Aniruddha Kembhavi and Ali Farhadi},
year = {2022},
eprint = {2212.08051},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2212.08051},
}
Simulates GelSight images from calibration examples (optical lookup + linear elastic deformation) instead of physics-based rendering, so tactile data can be generated quickly.用标定样本（光学查找表 + 线弹性形变）而不是基于物理的渲染来模拟 GelSight 图像，可快速生成触觉数据。
@misc{si2021taxim,
title = {{Taxim: An Example-based Simulation Model for GelSight Tactile Sensors}},
author = {Zilin Si and Wenzhen Yuan},
year = {2021},
eprint = {2109.04027},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2109.04027},
}
@misc{makoviychuk2021isaac,
title = {{Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning}},
author = {Viktor Makoviychuk and Lukasz Wawrzyniak and Yunrong Guo and Michelle Lu and Kier Storey and Miles Macklin and David Hoeller and Nikita Rudin and Arthur Allshire and Ankur Handa and Gavriel State},
year = {2021},
eprint = {2108.10470},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2108.10470},
}
JAX physics engine that runs thousands of environments in parallel on GPU/TPU together with the RL training loop, cutting locomotion training from hours to minutes.基于 JAX 的物理引擎，在 GPU/TPU 上与强化学习训练循环一起并行运行数千个环境，把运动控制训练从数小时缩短到数分钟。
@misc{freeman2021brax,
title = {{Brax: A Differentiable Physics Engine for Large Scale Rigid Body Simulation}},
author = {C. Daniel Freeman and Erik Frey and Anton Raichuk and Sertan Girgin and Igor Mordatch and Olivier Bachem},
year = {2021},
eprint = {2106.13281},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2106.13281},
}
Adds fast rigid-body physics and an interactive apartment dataset (ReplicaCAD) to Habitat, plus a home-rearrangement benchmark for mobile manipulators.为 Habitat 加入快速刚体物理与可交互公寓数据集 ReplicaCAD，并提出面向移动操作机器人的家居物品重排基准。
@misc{szot2021habitat,
title = {{Habitat 2.0: Training Home Assistants to Rearrange their Habitat}},
author = {Andrew Szot and Alex Clegg and Eric Undersander and Erik Wijmans and Yili Zhao and John Turner and Noah Maestre and Mustafa Mukadam and Devendra Chaplot and Oleksandr Maksymets and Aaron Gokaslan and Vladimir Vondrus and Sameer Dharur and Franziska Meier and Wojciech Galuba and Angel Chang and Zsolt Kira and Vladlen Koltun and Jitendra Malik and Manolis Savva and Dhruv Batra},
year = {2021},
eprint = {2106.14405},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2106.14405},
}
1,000 3D scans of real indoor spaces for Habitat; the main scene set behind ObjectNav and many embodied navigation benchmarks.1000 个真实室内空间的 3D 扫描，供 Habitat 使用；ObjectNav 等众多具身导航基准的主要场景集。
@misc{ramakrishnan2021habitat,
title = {{Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI}},
author = {Santhosh K. Ramakrishnan and Aaron Gokaslan and Erik Wijmans and Oleksandr Maksymets and Alex Clegg and John Turner and Eric Undersander and Wojciech Galuba and Andrew Westbury and Angel X. Chang and Manolis Savva and Yili Zhao and Dhruv Batra},
year = {2021},
eprint = {2109.08238},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2109.08238},
}
@misc{wang2020tacto,
title = {{TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensors}},
author = {Shaoxiong Wang and Mike Lambeta and Po-Wei Chou and Roberto Calandra},
year = {2020},
eprint = {2012.08456},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2012.08456},
}
@misc{zhu2020robosuite,
title = {{robosuite: A Modular Simulation Framework and Benchmark for Robot Learning}},
author = {Yuke Zhu and Josiah Wong and Ajay Mandlekar and Roberto Martín-Martín and Abhishek Joshi and Kevin Lin and Abhiram Maddukuri and Soroush Nasiriany and Yifeng Zhu},
year = {2020},
eprint = {2009.12293},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2009.12293},
}
A simulator with large-scale articulated objects (PartNet-Mobility) for interaction tasks like opening drawers.包含大规模铰接物体（PartNet-Mobility）的仿真器，适合开抽屉等交互任务。
@misc{xiang2020sapien,
title = {{SAPIEN: A SimulAted Part-based Interactive ENvironment}},
author = {Fanbo Xiang and Yuzhe Qin and Kaichun Mo and Yikuan Xia and Hao Zhu and Fangchen Liu and Minghua Liu and Hanxiao Jiang and Yifu Yuan and He Wang and Li Yi and Angel X. Chang and Leonidas J. Guibas and Hao Su},
year = {2020},
eprint = {2003.08515},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2003.08515},
}
DeepMind's Python stack on top of MuJoCo — bindings, PyMJCF and Composer for building models and tasks, the standard Control Suite benchmarks, plus locomotion and arm-and-bricks manipulation tasks.DeepMind 基于 MuJoCo 的 Python 工具栈：底层绑定、用于程序化建模与编写任务的 PyMJCF 和 Composer、作为标准基准的 Control Suite，以及运动控制和机械臂拼积木操作任务。
@misc{tassa2020dm,
title = {{dm_control: Software and Tasks for Continuous Control}},
author = {Yuval Tassa and Saran Tunyasuvunakool and Alistair Muldal and Yotam Doron and Piotr Trochim and Siqi Liu and Steven Bohez and Josh Merel and Tom Erez and Timothy Lillicrap and Nicolas Heess},
year = {2020},
eprint = {2006.12983},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2006.12983},
}
Open-source Gym-style benchmark for manipulating deformable objects such as cloth, rope and liquids; shows that high-dimensional, partially observable states make these tasks hard for standard RL.开源、Gym 接口的可变形物体操作基准（布料、绳子、液体等）；实验表明高维且部分可观测的状态让这类任务对常规强化学习方法很困难。
@misc{lin2020softgym,
title = {{SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation}},
author = {Xingyu Lin and Yufei Wang and Jake Olkin and David Held},
year = {2020},
eprint = {2011.07215},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2011.07215},
}
@misc{savva2019habitat,
title = {{Habitat: A Platform for Embodied AI Research}},
author = {Manolis Savva and Abhishek Kadian and Oleksandr Maksymets and Yili Zhao and Erik Wijmans and Bhavana Jain and Julian Straub and Jia Liu and Vladlen Koltun and Jitendra Malik and Devi Parikh and Dhruv Batra},
year = {2019},
eprint = {1904.01201},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/1904.01201},
}
@misc{tobin2017domain,
title = {{Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World}},
author = {Josh Tobin and Rachel Fong and Alex Ray and Jonas Schneider and Wojciech Zaremba and Pieter Abbeel},
year = {2017},
eprint = {1703.06907},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/1703.06907},
}
@misc{peng2017sim,
title = {{Sim-to-Real Transfer of Robotic Control with Dynamics Randomization}},
author = {Xue Bin Peng and Marcin Andrychowicz and Wojciech Zaremba and Pieter Abbeel},
year = {2017},
eprint = {1710.06537},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/1710.06537},
}
Photorealistic interactive indoor scenes where agents open, pick up and change objects; the base of many embodied-AI benchmarks such as ALFRED.具有可交互物体的逼真室内场景，智能体可开关、拾取和改变物体，是 ALFRED 等众多具身智能基准的基础。
@misc{kolve2017ai,
title = {{AI2-THOR: An Interactive 3D Environment for Visual AI}},
author = {Eric Kolve and Roozbeh Mottaghi and Winson Han and Eli VanderBilt and Luca Weihs and Alvaro Herrasti and Matt Deitke and Kiana Ehsani and Daniel Gordon and Yuke Zhu and Aniruddha Kembhavi and Abhinav Gupta and Ali Farhadi},
year = {2017},
eprint = {1712.05474},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/1712.05474},
}
Model-based toolbox for robotics - multibody dynamics with contact simulation, mathematical programming and trajectory optimization; used in the MIT manipulation and underactuated courses.面向机器人的基于模型的工具箱：带接触的多体动力学仿真、数学规划与轨迹优化；MIT 操作与欠驱动课程即使用它。
The long-standing open-source robot simulator of the ROS ecosystem — sensors, plugins and ROS 2 integration; the usual choice for testing full robot software stacks rather than training policies.ROS 生态中历史悠久的开源机器人仿真器，提供传感器、插件与 ROS 2 集成；通常用于测试完整的机器人软件栈，而非训练策略。
Dequan Ou, Austin Lopez, Quentin Missinne, Christophe De Wagter et al.
Abstract摘要
Event cameras offer low latency, high dynamic range, and milliwatt-level power consumption, making them ideal for agile robots such as tiny drones navigating through cluttered environments. However, existing simulators generally render photorealistic images before converting them into events, making large-scale reinforcement learning prohibitively slow. We propose an analytic event simulator that exploits what event cameras fundamentally measure - moving contrast edges - to synthesize event streams directly from scene geometry and ego-motion without visual rendering. Our simulator generates a …
@misc{ou2026rendering,
title = {{Rendering-Free Event Frame Simulation for Fast Reinforcement Learning of Autonomous Robot Navigation}},
author = {Dequan Ou and Austin Lopez and Quentin Missinne and Christophe De Wagter and Guido C. H. E. de Croon},
year = {2026},
eprint = {2610.37543},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37543},
}
Zhaoze Wang, Changxu Zhang, Tai Fei, Christopher Grimm et al.
Abstract摘要
Modern simulators can generate synthetic camera and LiDAR data with accurate annotations for downstream tasks. In contrast, automotive radar simulation remains less mature because it requires not only a 3D engine to model dynamic objects in the scenes but also electromagnetic material properties, antenna configurations, wave propagation mechanisms, and coherent signal synthesis. Existing frameworks are either proprietary or overly simplified, limiting the scalability of radar raw measurements. This paper presents AuRaSim, a ray-tracing-based automotive frequency-modulated continuous wave …
@misc{wang2026aurasim,
title = {{AuRaSim: A Ray-Tracing Based Automotive FMCW Radar Simulator for Dynamic Scenarios}},
author = {Zhaoze Wang and Changxu Zhang and Tai Fei and Christopher Grimm and Yi Jin and Claas Tebruegge and Markus Gardill},
year = {2026},
eprint = {2610.37403},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37403},
}
Ri-Zhao Qiu, Bowen Wen, Yifeng Jiang, Chen Tessler et al.
Abstract摘要
Real-to-Sim-to-Real (Real2Sim2Real) enables structured world representations for planning, interpretability, and safe execution. Most existing approaches rely on passive perception, which creates real2sim gaps due to imperfect observation. Inspired by how humans perceive actively through actions, we propose ActiveAlign, a framework that leverages active perception to align the Real2Sim2Real gap from interactions. ActiveAlign introduces two complementary modules: (i) ActiveAlign-Waving, which performs hand-eye calibration by exploiting active hand motions and differentiable rendering, and (ii) …
@misc{qiu2026activealign,
title = {{ActiveAlign: Aligning Egocentric Manipulation via Active Perception}},
author = {Ri-Zhao Qiu and Bowen Wen and Yifeng Jiang and Chen Tessler and Eric Heiden and Xue Bin Peng and Xiaolong Wang and Stan Birchfield and others},
year = {2026},
eprint = {2610.36859},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36859},
}
Low-cost arms such as the SO-101 have become the most common entry point to robot learning, yet no released policy performs spatially constrained, language-conditioned tasks on them out of the box, and existing SO-101 benchmarks run only on physical hardware with simple tasks and few held-out objects. We introduce SO-101 Bench, a simulated benchmark for language-conditioned spatial manipulation built on an Isaac Lab digital twin of a real SO-101 setup. It has four tasks scored as five families, 138 objects in seen, unseen-instance and unseen-class splits, 916 evaluation episodes with subsets …
Dianyi Yang, Mengya Liu, Zhumei Wang, Yu Liu et al.
Abstract摘要
Learning generalizable humanoid loco-manipulation remains bottlenecked by costly real-world demonstrations. Simulation provides a scalable alternative, but requires visually faithful, physics-ready scenes and executable whole-body motions. We present Zero-HLM, a real-to-sim-to-real framework that transforms smartphone captures and simulated source demonstrations into scalable training data for humanoid loco-manipulation. Gaussian-guided agentic real-to-sim reconstruction aligns visual and physical scenes; motion-environment co-adaptation uses rollout feedback to correct whole-body references …
@misc{yang2026zero,
title = {{Zero-HLM: Generalizable Humanoid Loco-Manipulation without Real Robot Data}},
author = {Dianyi Yang and Mengya Liu and Zhumei Wang and Yu Liu and Xiaochen Zhang and Yixuan Li and Le Ma and Yuhang Yang and others},
year = {2026},
eprint = {2610.35075},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35075},
}
Task success is not a sufficient measure of safety for embodied artificial intelligence (AI), because robots may complete objectives while violating safety requirements. Existing evaluations often emphasize performance and robustness, but lack a systematic route from documented safety hazards to executable, verifiable tests. We present EAST (Embodied AI Safety Testing), a digital-twin framework that converts literature-grounded safety evidence into fault-injection tests across perception, instructions, environments, actions, and timing. EAST combines an expert-reviewed safety-fault taxonomy …
@misc{tian2026east,
title = {{EAST: Embodied AI Safety Testing via Systematic Fault Injection in Digital Twins}},
author = {Yuhao Tian and Tao Li and Ling Zhou and Dave Towey and Zhaoxin Fan and Wenjun Wu and Rubing Huang and Tsong Yueh Chen},
year = {2026},
eprint = {2610.34358},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34358},
}
Construction robots operating near workers require navigation that can anticipate human motion while remaining robust to dynamic and unstructured site conditions. However, existing social navigation methods depend on hand-crafted interaction rules, labeled demonstrations, or robot-specific control policies, limiting their scalability and transferability in construction environments. We propose a real-time, LiDAR-only framework that learns interaction-aware path corrections from unlabeled demonstrations. Constant-velocity extrapolation generates residual learning targets, while a GNN models …
A dexterous hand can change an object's pose without setting it down. This survey covers 113 learning-based methods that attempt it, the hands they run on, the simulators they train in, and how they report results. The field has largely converged on one recipe: reinforcement learning on privileged state in a GPU simulator, distilled into a vision-only student for transfer to hardware. Almost half the corpus trains that way, nearly all of it in one simulator, whose own successors are barely used. A second and faster-growing branch learns from teleoperated demonstration instead, through …
Danilo Vucetic, Mohamad H. Danesh, Florence Clerc, Hsiu-Chin Lin et al.
Abstract摘要
Regularised reinforcement learning (RL) has provable robustness guarantees that should ease the transfer of policies from simulation to robots, known as sim-to-real. Yet, weakly regularised algorithms like Proximal Policy Optimisation (PPO) remain the de facto choice in sim-to-real pipelines, while the practical benefits of regularisation for transfer remain poorly characterised. In this work, we study the sim-to-real performance of commonly used RL algorithms, with the aim of characterising the effects of regularisation on robotic performance. To that end, we derive Soft MellowMax …
@misc{vucetic2026regularised,
title = {{Regularised RL improves sim-to-real transfer}},
author = {Danilo Vucetic and Mohamad H. Danesh and Florence Clerc and Hsiu-Chin Lin and Jonathan Tremblay},
year = {2026},
eprint = {2610.32855},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.32855},
}
Learned humanoid controllers are usually trained in one simulator and then checked in a second one before they go near hardware. We ask what that check says about command delay. We train motion-tracking policies for a 24-joint humanoid on five motions taken from monocular video and test them in an independent CPU MuJoCo harness, which we first verify against the training model. Every policy completes its motion in at least 26 of 30 episodes under the training randomization. With 40 ms of constant command delay, however, survival drops to 15% on average (0% - 45%), and the delay margin is only …
Tianling Xu, Weijie Wan, Honghao Su, Kailun Su et al.
Abstract摘要
Existing approaches for robot learning in simulation often require substantial manual effort to construct scenes and specify tasks, limiting training scalability and data diversity. Meanwhile, in-the-wild robot demonstrations naturally provide diverse and realistic observations of real-world interactions. Leveraging these demonstrations for simulation requires both high-fidelity visual reconstruction and physically consistent interactions. General-purpose agents offer flexible adaptation to diverse scenarios and strong reasoning about 3D spatial relationships, while specialized generative …
@misc{xu2026worldmirror,
title = {{WorldMirror: Agentic Real-to-Sim Generation of Interactive Worlds from In-the-Wild Robot Demonstrations}},
author = {Tianling Xu and Weijie Wan and Honghao Su and Kailun Su and Junyuan Tang and Huichi Zhou and Haozheng Zhang and Qi Xiong and others},
year = {2026},
eprint = {2610.31834},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31834},
}
Zikang Xiong, Weixin Li, Zhouchonghao Wu, Akshay Rangesh et al.
Abstract摘要
End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments. Its standard training recipe, however, is expensive across all existing paradigms: imitation learning requires fleet-scale driving logs whose behavior distribution only thinly covers the safety-critical tail; closed-loop reinforcement learning in a photorealistic simulator, while offering diverse behaviors, is bottlenecked by sample-efficiency, per-step cost of rendering, and a forward pass through a large vision backbone. Self-play in vectorized simulators changes the economics: …
@misc{xiong2026terratransfer,
title = {{TerraTransfer++: Demonstration-Free End-to-End Planner}},
author = {Zikang Xiong and Weixin Li and Zhouchonghao Wu and Akshay Rangesh and Saarth Bonde and Grantland Hall and Chen Tang and Yihan Hu and others},
year = {2026},
eprint = {2610.30319},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30319},
}
Yipeng Lu, Youqi Liao, Zhenyu Wang, Yuning Peng et al.
Abstract摘要
The scarcity of high-quality annotated visual data remains a critical bottleneck for autonomous-driving perception. Simulators provide training data at scale, but the limited realism of synthetic videos compromises perception performance in real-world applications. To address this limitation, we propose NeuralSRender, a sim-to-real video translation framework that enhances the photorealism of synthetic videos while maintaining geometric consistency with the source scenes. Built on a pretrained video diffusion transformer (DiT), NeuralSRender uses a lightweight projector to inject geometric …
@misc{lu2026neuralsrender,
title = {{NeuralSRender: G-Buffer Guided Video Diffusion for Sim-to-Real Rendering}},
author = {Yipeng Lu and Youqi Liao and Zhenyu Wang and Yuning Peng and Zhen Dong},
year = {2026},
eprint = {2610.29200},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.29200},
}
We present HyperDrone, an extremely fast quadrotor simulator focused on high-performance batch rendering that is more than 20x faster than existing state-of-the-art batch quadrotor simulators. HyperDrone combines highly optimized multirotor dynamics with HyperLux, a new ray-tracing engine that supports Nvidia OptiX, Vulkan, Apple Metal and WebGPU backends that takes full advantage of modern ray-tracing hardware, capable of running both components entirely on the GPU. HyperLux renders RGB, depth, semantic/instance segmentation, surface normals and optical flow observations at configurable …
@misc{eschmann2026hyperdrone,
title = {{HyperDrone: Simulation-to-Reality Transfer at Millions of Frames per Second}},
author = {Jonas Eschmann and Giuseppe Loianno},
year = {2026},
eprint = {2610.28950},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28950},
}
Jiahui Yang, Jason Jingzhou Liu, Ray Muxin Liu, Andrew E. Wang et al.
Abstract摘要
Mobile manipulators must adapt not only to objects, but also to the spatial context in which they appear: where to position the base, how to manipulate, and where to look while navigating surrounding scene geometry. To address this, we present COWBOY, a scalable sim-to-real framework for learning such contextual whole-body manipulation. On whole-body dexterous grasping, COWBOY transfers zero-shot to hundreds of objects and diverse in-the-wild scenes, spanning cluttered tabletops, shelves, drawers, and other unstructured settings. Rather than learning scene adaptation directly with RL in …
@misc{yang2026cowboy,
title = {{COWBOY: A Scalable Sim-to-Real Framework for Learning Contextual Whole-Body Manipulation}},
author = {Jiahui Yang and Jason Jingzhou Liu and Ray Muxin Liu and Andrew E. Wang and Kenneth Shaw and Deepak Pathak and Ruslan Salakhutdinov},
year = {2026},
eprint = {2610.28948},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28948},
}
Lunar surface exploration increasingly relies on autonomous rovers that must perceive and traverse unstructured terrain under extreme illumination conditions. Near the lunar poles, low sun elevations produce long, high-contrast shadows that challenge stereo vision, terrain segmentation, and localization. Developing perception and navigation algorithms for these conditions requires large amounts of sensor data with accurate ground truth, which field campaigns and analog facilities can only provide at high cost and with limited variety. This paper presents MARE (Modular Autonomy and Rendering …
@misc{castillaarquillo2026mare,
title = {{MARE: An Open-Source Lunar Rover Simulator with Shadow Ground Truth for Perception and Autonomy}},
author = {Ra\'ul Castilla-Arquillo and Miguel Olivares-Mendez},
year = {2026},
eprint = {2610.28465},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28465},
}
Charbel Abi Hana (Idealworks), Rayan Zaitouni (Idealworks), Kameel Amareen (Idealworks), Mohamad Mostafa (Idealworks) et al.
Abstract摘要
Classical robot navigation stacks are reliable but often require re-engineering for each new robot type and site. Learning-based navigation shifts this effort from hand-tuned parameters to data generation and training, where a policy adapts to each new embodiment or facility by fine-tuning in simulation. Whether policies adapted this way stay reliable on real industrial robots has seen little systematic testing. We study this for NVIDIA's X-Mobility and COMPASS workflow for training learned navigation policies on Idealworks robots. In simulation, we extend the COMPASS benchmark with two …
@misc{idealworks2026physical,
title = {{Physical AI for Intralogistics: Benchmarking the NVIDIA Sim-to-Real Ecosystem for Cosmos-Enhanced COMPASS Navigation on Industrial Mobile Robots}},
author = {Charbel Abi Hana (Idealworks) and Rayan Zaitouni (Idealworks) and Kameel Amareen (Idealworks) and Mohamad Mostafa (Idealworks) and Hasan Sayour (Idealworks) and Wei Liu (NVIDIA) and Yan Chang (NVIDIA) and Mihir Acharya (NVIDIA) and others},
year = {2026},
eprint = {2610.28432},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28432},
}
Synchronization between a digital twin (DT) and its physical robot is critical for effective teleoperation, as mismatches directly degrade operator awareness and task performance. Existing synchronization metrics are predominantly time-based and rarely capture the spatial consequences of delayed information during motion. In this paper, we introduce Kinematic Age of Information (K-AoI), a spatio-temporal metric that weights AoI by instantaneous Cartesian velocity, translating temporal staleness into physically meaningful spatial divergence. We implement and evaluate K-AoI on a DT …
@misc{tian2026k,
title = {{K-AoI: A Spatio-Temporal Synchronization Metric for Digital Twin-based Teleoperation in Dynamic Network Environments}},
author = {Lu Tian and Zhenhui Yuan and Zexin Deng and Wenjuan Zhang},
year = {2026},
eprint = {2610.26201},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.26201},
}
Simulation-to-real driving video diffusion models can translate photorealistic scenes, but they typically represent pedestrians and cyclists, vulnerable road users (VRUs), as generic object classes. This is insufficient for vulnerable road users whose safety-relevant appearance depends on articulated pose and identity. We present BodyBridge and Grounded Body Layers (GBL), a simulator-derived, pixel-aligned representation that combines scene geometry, canonical per-pixel body correspondence, and exact VRU masks. GBL provides explicit articulation to the video diffusion model, emphasizes VRU …
Antonia Bronars, Younghyo Park, Kushal Kedia, Pulkit Agrawal
Abstract摘要
Every robot already carries high-resolution contact sensors: its own joints. Legged robots and humanoids exploit them to walk, run, and track whole-body motions without vision, but there the contacts are large and forceful. Is joint sensing also enough for precise, contact-rich assembly, where the robot must find a millimeter-scale opening, mesh gear teeth, or catch a thread through light contact? We study this question by training blind assembly policies that observe only a short history of joint positions, velocities, and their own actions, without visual, force, torque, or tactile sensing. …
Al Bashir, Partho Ghose, Yibin Wang, Robert G. Hardin et al.
Abstract摘要
Robotic harvesting requires precise manipulation to ensure accurate positioning of the end-effector, especially in a constraint workspace. This study introduces the Path-Safe Approach Proximal Policy Optimization (PSA-PPO), a target-conditioned local reaching framework designed for robotic harvesting of nutrient film technique (NFT)-hydroponic lettuce in greenhouses. The proposed PSA-PPO preserves the standard clipped PPO objective while introducing a harvesting-oriented policy formulation that integrates target-conditioned observations, corridor-aware spatial constraints, …
@misc{bashir2026sim,
title = {{Sim-to-Real Reinforcement Learning for Path-Safe Target Reaching in Robotic Harvesting of Hydroponic Lettuce}},
author = {Al Bashir and Partho Ghose and Yibin Wang and Robert G. Hardin and Joshua Peeples and Genhua Niu and Azlan Zahid},
year = {2026},
eprint = {2610.22337},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.22337},
}
Zhou Daniel Hao, Umer Masood Chaudry, Robert Shaw, Andrew Cheney et al.
Abstract摘要
Growing interest in in-space servicing, manufacturing, and assembly has renewed attention on reliable metal joining for modular construction and on-orbit repair. Robotic welding is promising, but reduced-gravity and low-pressure operation remains poorly characterised and flight-like tests are limited. The UK Space Agency-supported ISPARK project is developing a physics-grounded digital twin for structured process de-risking. This paper evaluates a model-based prototype combining a finite-volume thermal weld-pool model, Gaussian-process (GP) and neural-network surrogates, and a simulated …
@misc{hao2026de,
title = {{De-Risking Robotic Welding for In-Space Metal Joining Through Physics-Grounded Digital Twins}},
author = {Zhou Daniel Hao and Umer Masood Chaudry and Robert Shaw and Andrew Cheney and Nick Ludford and Mark Sims and Hongbiao Dong},
year = {2026},
eprint = {2610.21939},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.21939},
}
Jiawei Zhang, Chang Liu, Litao Deng, Tao Zhang et al.
Abstract摘要
Scaling embodied robot learning requires large and diverse manipulation datasets, but collecting real-world demonstrations remains expensive in human effort and robot time. Simulation provides a scalable alternative, yet converting a real workspace into a robot-learning-ready environment still requires substantial manual effort for object extraction, asset processing, coordinate calibration, and trajectory specification. We present AutoRSR, a highly automated real-to-sim-to real framework that converts scanned real-world workspaces into reusable embodied simulation assets and scalable …
@misc{zhang2026autorsr,
title = {{AutoRSR: Automated Real-to-Sim-to-Real Data Generation for Scalable Robot Learning}},
author = {Jiawei Zhang and Chang Liu and Litao Deng and Tao Zhang and Qin Jin},
year = {2026},
eprint = {2610.20336},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.20336},
}
Technologies, such as co-simulation, in particular, digital twins (DTs) are used for comprehensive and reliable prototyping, verification and validation (V&V) of control systems. There, the combination of feedback loops across multiple realities (e.g., simulations, physical setups) enables various forms of experimentation with robots and autonomous systems. However, the handling of multiple realities requires a more specific support by the modelling techniques used in control engineering. To streamline the latter in the system life cycle, we propose a framework for mixing (i.e., fusing, …
@misc{soydan2026mixed,
title = {{Mixed-Reality Feedback Control: Towards Blending of Control and Digital Twinning}},
author = {Melisa Soydan and Mario Gleirscher},
year = {2026},
eprint = {2610.19759},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.19759},
}
Sim-and-real co-training is an effective way to scale robot learning by augmenting limited real-world demonstrations with simulation data. However, existing approaches typically treat simulators as fixed data sources, relying on uniform generation, offline retrieval, or manual parameter alignment by domain experts. We formulate \textit{Active Generative Simulation}: optimizing a simulator's data-generation distribution to maximize downstream real-world performance. To solve this problem, we propose MetaGenSim, a meta-learning framework that trains an auxiliary rater network via meta-gradients …
@misc{ma2026active,
title = {{Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training}},
author = {Liqian Ma and Vaibhav Saxena and Utkarsh Aashu Mishra and Shuo Cheng and Yongxin Chen and Danfei Xu},
year = {2026},
eprint = {2610.19479},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.19479},
}
Yushen Hu, Harshvardhan Chandirasekar, Apurv Kushwaha, Jonathan King et al.
Abstract摘要
Digital twins are increasingly used in robotics to simulate interactions, predict action outcomes, and support planning. However, constructing accurate digital twins requires precise estimates of physical properties such as mass, friction, and center of mass, which are often difficult to obtain in real-world environments. We propose probabilistic digital twins, which represent uncertain physical properties as probability distributions rather than fixed values. By simulating actions across these distributions, the twin predicts a range of possible outcomes and their likelihoods. These …
@misc{hu2026probabilistic,
title = {{Probabilistic Physics Digital Twin for Human-Robot Interaction}},
author = {Yushen Hu and Harshvardhan Chandirasekar and Apurv Kushwaha and Jonathan King and Zhu-Tian Chen},
year = {2026},
eprint = {2610.19336},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.19336},
}
Nolan Fey, Khai Nguyen, Yash Prabhu, Gabriel B. Margolis et al.
Abstract摘要
Humanoid policies trained in simulation can transfer qualitatively to hardware while failing to transfer the force interactions learned during training, compromising torque efficiency, load regulation, and robustness to impacts. This failure can arise from actuator sim-to-real gaps because forces depend on actuator torques. We introduce Contact-UAN, a residual world model that preserves a simulator's physics priors while correcting actuator dynamics to align simulated rollouts with recorded hardware rollouts. For humanoid motion with contact, this requires contact-consistent replay: if …
@misc{fey2026contact,
title = {{Contact-UAN: Unsupervised Actuator Networks as a Residual World Model for Humanoid Control}},
author = {Nolan Fey and Khai Nguyen and Yash Prabhu and Gabriel B. Margolis and Pulkit Agrawal},
year = {2026},
eprint = {2610.18991},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.18991},
}
Yunfu Deng, Sangwoo Shin, Brahma S. Pavse, Josiah P. Hanna
Abstract摘要
Evaluating robot policies before deployment typically requires producing new trajectories through real-world execution, simulators, or learned world models. These rollout-based approaches impose infrastructure requirements; simulator and world-model evaluators additionally face prediction errors that can compound over the evaluation horizon. In this paper, we study off-policy evaluation as an alternative to rollout-based robot-policy evaluation methods. Specifically, we introduce RoboFQE, an evaluator that estimates the discounted expected return of a target policy from offline robot …
@misc{deng2026robofqe,
title = {{RoboFQE: Rollout-Free Evaluation of Visual Robot Policies from Offline Trajectories}},
author = {Yunfu Deng and Sangwoo Shin and Brahma S. Pavse and Josiah P. Hanna},
year = {2026},
eprint = {2610.18699},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.18699},
}
Shangqing Zhou, Zhennan Jiang, Ning Ma, Xin Fu et al.
Abstract摘要
Action-conditioned world models can reduce costly real-robot interaction in reinforcement learning for Vision-Language-Action models (VLA-RL), but hallucinations can mislead policy optimization. Extending them to richer VLA interfaces and long-horizon tasks raises challenges in spatiotemporal and cross-modal consistency, exploration-reliability balance in RL, and compounding rollout errors. We present WoVR+, extending WoVR's hallucination-aware WM-RL framework by redesigning the learned environment and imagined interaction protocol: A VLA-interface-aligned world model provides multi-view …
@misc{zhou2026wovr,
title = {{WoVR+: World Model as Reliable Simulator for Versatile VLA Policy Optimization with RL}},
author = {Shangqing Zhou and Zhennan Jiang and Ning Ma and Xin Fu and Yupeng Zheng and Shuai Tian and Huangrui Li and Jiahao Liu and others},
year = {2026},
eprint = {2610.17922},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.17922},
}
Hiroki Nishizawa, Takehiko Ohkawa, Nicola Scianca, Tatsuya Matsushima et al.
Abstract摘要
Evaluating VLA policies on real robots is costly, time-consuming, and difficult to reproduce, making real-to-sim simulation essential for scalable evaluation. While 3D Gaussian Splatting (3DGS) enables rapid real-to-sim scene generation, the reconstruction quality required for reliable policy evaluation remains unquantified. In particular, policies relying on wrist-mounted fisheye cameras (e.g., YUBI handheld gripper data) are heavily exposed to visual artifacts (holes, floaters) in wide-angle views, directly degrading task performance. To address this, we present YUBI-Real2Sim, a unified …
Haozhe Lou, Runyi Yang, Wanqi Zhong, Caiyun Liu et al.
Abstract摘要
Cutting a piece of food looks effortless in human hands. For a robot, however, every cut is a high-contact interaction with a deformable, elasto-plastic object whose behavior changes with pose, contact mode, and cutting stage. Collecting real-world data is expensive and wasteful: each trial typically contaminates or damages the food, so obtaining scalable real world data for dexterous cutting is expensive and difficult. Simulation is an appealing alternative, but a large sim-to-real gap exists in both physics and perception. This gap is especially severe for topology-changing interactions …
@misc{lou2026dexninja,
title = {{DexNinja: Learning Robust Dexterous Cutting Policy with a Real-to-Sim-to-Real Data Engine}},
author = {Haozhe Lou and Runyi Yang and Wanqi Zhong and Caiyun Liu and Yicheng Liu and Wenhao Liu and Wenlong Ma and Jiawei Xia and others},
year = {2026},
eprint = {2610.17208},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.17208},
}
Wenzhan Li, Yiran Qin, Heng Zhou, Huirui Wang et al.
Abstract摘要
Vision-Language-Action (VLA) models are often pretrained via imitation learning, making reinforcement learning (RL) post-training important when deployment moves beyond demonstration coverage. Existing refinement paradigms expose a reliability--efficiency tension: model-free refinement avoids simulator bias but is interaction-intensive, whereas model-based refinement can exploit unreliable imagination even as its simulator adapts. We propose Model-Evolving Reinforcement Learning (MERL), a trust-calibrated framework that couples policy optimization with an online-updated simulator comprising a …
@misc{li2026trust,
title = {{Trust-Calibrated VLA Policy Post-Training with Evolving Imagination}},
author = {Wenzhan Li and Yiran Qin and Heng Zhou and Huirui Wang and Yulan Guo and Ruimao Zhang},
year = {2026},
eprint = {2610.16517},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.16517},
}
Deep reinforcement learning has shown strong potential for vision-based autonomous driving, but policies trained in simulation often suffer severe performance degradation when deployed in the real world. Existing sim-to-real methods commonly address this gap through image-transfer-based domain adaptation or policy-agnostic domain randomisation. However, the former requires laborious acquisition of target-domain data and is often limited to the deployment domain, whereas the latter introduces unguided perturbations that may not reflect the underlying sim-to-real discrepancy, leading to …
@misc{yao2026xai,
title = {{XAI-Guided Fourier Domain Generalisation for Pure-Simulation Sim-to-Real Deep Reinforcement Learning}},
author = {Wei Yao and Chuyao Wang and Ziwei Wang and Nabil Aouf},
year = {2026},
eprint = {2610.16478},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2610.16478},
}
Kota Tsuruhara, Tsung-Yuan Chen, Kento Kawaharazuka
Abstract摘要
Humans expand their locomotion capability by wearing appropriate footwear, such as hiking boots for mountains or skates for ice. Footwear changes the ground-contact interface and enables traversal of otherwise difficult terrain. In contrast, locomotion controllers are typically trained for a fixed morphology, and their performance can degrade when shoes alter foot geometry, mass, and friction. We propose SHOES, a sample-efficient framework for adapting locomotion policies to footwear-induced morphology changes. SHOES freezes a simulation-trained nominal policy and fine-tunes only lightweight, …