RECAP trains a VLA with RL via advantage conditioning on demos, on-robot rollouts and teleop corrections; reports it more than doubles throughput on hard tasks like laundry folding and espresso making.RECAP 通过优势条件化，用演示、机器人自主运行数据和遥操作纠正对 VLA 做强化学习；论文报告在叠衣服、做意式咖啡等难任务上吞吐量提升一倍以上、失败率约减半。
@misc{intelligence2025vla,
title = {{π*0.6: a VLA That Learns From Experience}},
author = {Physical Intelligence and Ali Amin and Raichelle Aniceto and Ashwin Balakrishna and Kevin Black and Ken Conley and Grace Connors and James Darpinian and Karan Dhabalia and Jared DiCarlo and Danny Driess and Michael Equi and Adnan Esmail and Yunhao Fang and Chelsea Finn and Catherine Glossop and Thomas Godden and Ivan Goryachev and Lachy Groom and Hunter Hancock and Karol Hausman and Gashon Hussein and Brian Ichter and Szymon Jakubczak and Rowan Jen and Tim Jones and Ben Katz and Liyiming Ke and Chandra Kuchi and Marinda Lamb and Devin LeBlanc and Sergey Levine and Adrian Li-Bell and Yao Lu and Vishnu Mano and Mohith Mothukuri and Suraj Nair and Karl Pertsch and Allen Z. Ren and Charvi Sharma and Lucy Xiaoyang Shi and Laura Smith and Jost Tobias Springenberg and Kyle Stachowicz and Will Stoeckle and Alex Swerdlow and James Tanner and Marcel Torne and Quan Vuong and Anna Walling and others},
year = {2025},
eprint = {2511.14759},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2511.14759},
}
HIL-SERL — real-world RL with human corrections, reported to reach near-perfect success on precise assembly and dynamic tasks after a few hours of training.HIL-SERL：结合人类纠正的真机强化学习，报告称在精密装配与动态任务上经数小时训练即可接近满分成功率。
@misc{luo2024precise,
title = {{Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning}},
author = {Jianlan Luo and Charles Xu and Jeffrey Wu and Sergey Levine},
year = {2024},
eprint = {2410.21845},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.21845},
}
The clipped policy-gradient method behind most GPU-parallel sim-to-real locomotion and dexterity training.采用截断目标的策略梯度方法，是大多数 GPU 并行仿真训练（运动控制、灵巧操作）背后的算法。
Tests six flow-matching VLAs and reports that one-step Euler action generation largely matches multi-step sampling at scale — a useful check before adding special one-shot objectives to speed up control (CoRL 2026).在六个基于流匹配的 VLA 上实验，报告单步 Euler 生成动作在规模足够时与多步采样效果基本相当——在为提速引入专门的单步生成目标之前值得先看（CoRL 2026）。
@misc{sujit2026when,
title = {{When One Step Is Enough: An Empirical Study of Euler Integration in Flow-Matching Policies}},
author = {Shivakanth Sujit and Luca Nunziante and Rousslan Fernand Julien Dossa and Kai Arulkumaran},
year = {2026},
eprint = {2610.30735},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.30735},
}
Instead of one fixed set of VLA weights, generates context-dependent low-rank updates for both the vision-language and action modules; reports gains up to 16.3% on LIBERO/Meta-World and up to 14% over π0.5 on real tasks.不再让 VLA 在所有交互情境下共用固定参数，而是按当前情境为视觉语言模块和动作模块生成耦合的低秩参数更新；报告在 LIBERO/Meta-World 上最高提升 16.3%，真机上相对 π0.5 最高提升 14%。
@misc{liu2026morphact,
title = {{MorphAct: Robot Policies Morphing with the Worlds They Act In}},
author = {Ziyan Liu and Jiting Liu and Weihong Zhong and Wenda Yu and Yeqiu Chen and Jason Li and Xiaojiang Peng and Shuyuan Hu and Kai Wang and Bo Zhao},
year = {2026},
eprint = {2610.20019},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.20019},
}
Practical warning: fine-tuning VLA weights in bf16 without an fp32 master copy rounds away AdamW updates below 1/512 of a weight; finds this recipe in 9 of 12 open VLA codebases and 79%+ of updates lost on LIBERO.实用警示：VLA 权重以 bf16 微调且不保留 fp32 主副本时，小于权重 1/512 的 AdamW 更新会被舍入抹掉；报告 12 个开源 VLA 代码库中 9 个的默认或文档配方如此训练，LIBERO 实验中至少 79% 的更新被抹除。
Pretrains a 4B VLA on 3D-grounded latent actions mined from 30K hours of human, robot and sim video, then adapts to real robot actions. Reports 98.8% on LIBERO and 88.6% on RoboTwin 2.0.从 3 万小时人类、机器人与仿真视频中学习带 3D 约束的连续潜在动作（RynnLAM），用于预训练 4B 参数 VLA，再适配到具体机器人动作空间。报告 LIBERO 98.8%、RoboTwin 2.0 平均 88.6%。
@misc{zhao2026rynnvla,
title = {{RynnVLA-Latent: Scaling Robotic Manipulation Foundation Models with 3D-Grounded Latent Actions}},
author = {Haoyu Zhao and Xingyue Zhao and Zhian Su and Bohan Hou and Yinzhou Tang and Fengyi Wang and Kehan Li and Yuming Jiang and Siteng Huang and Xin Li and Zhenyu Zeng and Mingxiu Chen and Deli Zhao and Zhongyu Li},
year = {2026},
eprint = {2610.34356},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34356},
}
Meta-agent that improves a robot's harness by exploring: keeps a lineage graph of harness versions and runs new rollouts to probe failures before editing. Reports more success with fewer rollouts than static reflection.通过主动探索改进机器人 harness 的元智能体：维护 harness 版本谱系图，先主动生成新回放探查失败与恢复行为再修改；报告在 LIBERO-Pro、RoboCasa、RoboTwin 上比静态反思用更少仿真回放取得更高成功率。
@misc{zhang2026vista,
title = {{VISTA: Active Exploration for Self-Improvement in Embodied Agents}},
author = {Zhang Zhang and Shengjun Zhang and Simin Huang and Min Chen and Chong Xia and Xin Fei and Yueqi Duan},
year = {2026},
eprint = {2610.35051},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35051},
}
Latent action model that fuses synchronized camera views to resolve depth and occlusion. Reports 17% better 6D action decodability than single-view and +13.5% policy success (vs +6.2%); human-hand demos help on hardware.融合多路同步相机视角的潜在动作模型（LAM），用交叉注意力解决深度与遮挡问题；报告 6D 动作可解码性比单视角高 17%，下游策略成功率提升 13.5%（单视角为 6.2%），真机上用人手演示也比纯动作策略高 20%。
@misc{hausdorfer2026multi,
title = {{Multi-View Latent Action Pretraining for Cross-Embodiment Robot Learning from Videos}},
author = {Oliver Hausdörfer and Felix Minzenmay and Mani Majd and Maximilian P. Christof and Shiyuan Zhang and Angela P. Schoellig},
year = {2026},
eprint = {2610.27794},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.27794},
}
VLA that reasons in discrete motion tokens (a chain of dynamics) shared across embodiments; predicted futures are scored against what actually happened, so action-free human videos give RL signal. Sim and real tests.VLA 用跨本体共享的离散运动 token 链（Chain-of-Dynamics）表达动作思考；预测每条链导致的未来并与真实结果比对来强化，使没有动作标注的人类视频也能提供监督。在仿真与真机上验证人到机器人的迁移。
@misc{kim2026thinkworld,
title = {{ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning}},
author = {Chaehyun Kim and Chi-Pin Huang and Jisoo Kim and Min-Hung Chen and Yu-Chiang Frank Wang and Seungryong Kim and Fu-En Yang},
year = {2026},
eprint = {2610.37666},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37666},
}
CTR retimes the two arms in bimanual demos at training time while keeping task-required constraints, so the policy stops using the other arm's progress as a shortcut cue.CTR 在训练时对双臂示教中两只手臂的动作时序重新组合，同时保留任务必需的约束，避免策略把另一只手臂的进度当作捷径线索。
@misc{hu2026learn,
title = {{Learn the Task, Not the Schedule: Reducing Spurious Cross-Arm Coupling in Bimanual Imitation Learning}},
author = {Shuyuan Hu and Deng-Ke Wu and Yifan Jin and Qianhang Luo and Mingchen Ju and Kai Ye and Bo Zhao},
year = {2026},
eprint = {2610.26507},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.26507},
}
One two-hand policy (PPO + BC on its own verified successes) tracks retargeted human hand-object motions; ships BiHOI (1,575 sequences, 229 objects). Reports 86.8% on unseen trajectories, 57.3% on new objects.用一个双手跟踪策略（PPO 结合对自身成功执行的行为克隆）复现重定向后的人手-物体交互动作，并发布 BiHOI 数据集（1575 段双手序列、229 个物体）。论文报告在未见轨迹上成功率 86.8%、新物体上 57.3%。
@misc{sun2026donic,
title = {{DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations}},
author = {Mingze Sun and Ziang Cao and Yuan Fei and Qiankuan Tao and Zihan Wang and Tianqi Liu and Junhao Chen and Haiwen Diao and Fangzhou Hong and Zhaoxi Chen and Ziwei Liu},
year = {2026},
eprint = {2610.29997},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.29997},
}
Mobile dexterous grasping learned in sim: a whole-body controller does reaching and active vision, local RL experts the object, all distilled into one point-cloud policy; reports zero-shot real transfer.在仿真中学习移动灵巧抓取：全身控制器负责底盘-手臂协同到达与主动视觉，局部强化学习专家负责物体交互，二者蒸馏成一个基于点云的视觉策略；报告可零样本迁移到真实世界的数百个物体与场景。
@misc{yang2026cowboy,
title = {{COWBOY: A Scalable Sim-to-Real Framework for Learning Contextual Whole-Body Manipulation}},
author = {Jiahui Yang and Jason Jingzhou Liu and Ray Muxin Liu and Andrew E. Wang and Kenneth Shaw and Deepak Pathak and Ruslan Salakhutdinov},
year = {2026},
eprint = {2610.28948},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28948},
}
ARPG pretrains an action encoder on human manipulation data, then trains a 130M generative policy in its space. Reports bimanual dexterous success up from 10.1% (π0.5) to 44.7% on DexJoCo.ARPG 把动作表征学习与动作生成解耦：先在大规模人类操作数据上预训练动作编码器，再在其表征空间训练 1.3 亿参数的生成策略。报告在 DexJoCo 双手灵巧任务上平均成功率从 π0.5 的 10.1% 提升到 44.7%。
@misc{xu2026scaling,
title = {{Scaling Dexterous Manipulation via Action Representation Pretraining}},
author = {Yanjinghao Xu and Jiexin Zhang and Huan Zhao and Jianjia Mo and Zhenghao Yin and Dingkang Liang and Xiang Bai and Han Ding},
year = {2026},
eprint = {2610.34453},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34453},
}
Language-to-dexterous-manipulation as modulated tracking: a distilled generalist tracker plus an RL-trained modulator that keeps adjusting the reference, object latent and position target; zero-shot sim-to-real.把语言条件灵巧操作变成“可调制的跟踪”：蒸馏得到的通用跟踪器加 RL 训练的调制器，在执行中持续修正参考轨迹、物体隐表征与位置目标，抑制误差累积；零样本迁移到真机。
@misc{adalibieke2026adadextrack,
title = {{AdaDexTrack: Dynamic Modulation for Adaptive and Generalizable Dexterous Manipulation Tracking}},
author = {Jianibieke Adalibieke and Qianwei Han and Xueyi Liu and Yuzhe Qin and Li Yi},
year = {2026},
eprint = {2610.33852},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.33852},
}
Foresight without predicting future frames: a training-only head forecasts the next action chunk and backpropagates through the policy's latent. No inference cost; reports up to +14 pts on mm-precision real tasks.不预测未来图像也能获得“预见性”：仅在训练时用级联头预测下一个动作块，并把损失回传到在线策略的隐状态，推理零额外开销；报告在耳机装配、穿鞋带等毫米级真机任务上最高提升 14 个百分点。
@misc{yang2026adapolicy,
title = {{AdaPolicy: Action-Driven Anticipatory Robot Policies via Next-Chunk Prediction}},
author = {Fan Yang and Yuting Su and Chen Zhao and Fugui Fan and Peiguang Jing},
year = {2026},
eprint = {2610.34280},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34280},
}
DISCO runs progressive-widening MCTS in GPU-parallel simulation, with a learned policy proposing continuous actions, then distills plans into a policy-value model used without search. Reports wins over BC, RL and search.DISCO 在 GPU 并行物理仿真中用渐进展宽 MCTS 搜索长程重排计划，由学习的策略提出连续动作、价值函数打分，并周期性把搜索结果蒸馏回策略-价值模型；部署时无需仿真搜索。报告在仿真与真机重排任务上优于行为克隆、强化学习与搜索基线（CoRL 2026 Oral）。
@misc{saha2026long,
title = {{Long-Horizon Multi-Object Rearrangement via Distilled Tree Search over Continuous Actions}},
author = {Kallol Saha and Yining Hong and Amber Li and Yejin Choi and Maxim Likhachev and David Held},
year = {2026},
eprint = {2610.25785},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.25785},
}
@misc{strong2026real,
title = {{Real2Sem2Real: 3D Semantic Augmentations for Robot Learning from a Single Human RGB Demonstration}},
author = {Matthew Strong and Ankush K. Dhawan and Julien Poffet and Giuse Pham and Jinho So and Monroe Kennedy},
year = {2026},
eprint = {2610.28880},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28880},
}
Sim-only RL recipe for in-hand object pose reaching on a LEAP hand, zero-shot to hardware; reports 95.2% of 4,800 unseen goals reached and a tunable residual safety filter that cuts drops by 75%.在仿真中无需人类数据训练 LEAP 灵巧手的手内物体位姿到达策略，零样本迁移到真机；报告在 4800 个未见目标上 95.2% 到达，可调节的残差安全滤波器使掉落减少 75%。
@misc{chuang2026pushing,
title = {{Pushing Dexterity and Safety for In-Hand Object Pose Reaching That Works Remarkably Well}},
author = {Ian Chuang and Hiya Shah and Vamsi Chilamkur and Iman Soltani and Claire Tomlin},
year = {2026},
eprint = {2610.37652},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37652},
}
Learns history-aware progress rewards for long-horizon manipulation so missing prerequisites are not rewarded, then refines policies from BC to online RL. Reports 31.8% on 18 held-out RLBench tasks (+9.2 pts).为长时程操作学习依赖执行历史的进度奖励，避免跳过前置步骤也得高分，再从行为克隆过渡到在线强化学习微调策略；报告在 18 个未参与预训练的 RLBench 任务上平均 31.8%（高出最强基线 9.2 个百分点），真机 57.5%。
@misc{chen2026robolongr,
title = {{RoboLongR: Causally Consistent Rewards Teach Robots to Progressively Master Long-Horizon Manipulation}},
author = {Zixuan Chen and Lin Cao and Fenggang Zhang and Haotong Zhang and Yangtao Chen and Jieqi Shi and Jing Huo and Yang Gao},
year = {2026},
eprint = {2610.33863},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.33863},
}
Builds pseudo-bimanual demos from single-arm data by sagittal-plane mirroring plus LLM-written coordination constraints. On AgileX ALOHA it reports 2.11× cups collected vs. a single-arm baseline.将单臂示范沿矢状面镜像（观测、本体感觉与动作），再由 LLM 生成时空协调约束，构造伪双臂示范；在 AgileX ALOHA（RoboTwin 仿真与真机）上报告收集杯子数达到单臂基线的 2.11 倍。
@misc{wu2026robostd,
title = {{RoboSTD: Zero-Shot Single-to-Dual Transfer via Sagittal Mirroring for Bimanual Learning}},
author = {Hongyu Wu and Chong Liu and Shaoxuan Xie and Shaqi Luo and Hanyu Feng and Shihan Wu and Guocai Yao},
year = {2026},
eprint = {2610.36685},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36685},
}
Rather than distilling per-motion experts or brute-forcing one RL policy, trains a humanoid tracking policy jointly with one differentiable world model of shared dynamics, fit from simulator logs (CoRL 2026 oral).SuperMimic 不再蒸馏逐动作专家或用大算力硬训单一 RL 策略，而是把人形机器人动作跟踪策略与一个从仿真日志拟合的、共享动力学的可微世界模型联合训练（CoRL 2026 口头报告）。
@misc{liao2026supermimic,
title = {{SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model}},
author = {Qiayuan Liao and Yiyang Shao and Junfeng Long and Kevin Zakka and Roberto Horowitz and Pieter Abbeel and Koushil Sreenath},
year = {2026},
eprint = {2610.28480},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28480},
}
Fine-tunes humanoid motion-tracking policies online on the real robot: a proprioceptive neural dynamics model learned from real data supplies gradients, no motion capture needed. Shown on Unitree G1.在真机上在线微调人形机器人动作跟踪策略：用真实数据训练的本体感知神经动力学模型提供可反传梯度，无需动捕系统。在 Unitree G1 上验证对多种扰动的快速适应。
@misc{chen2026humanoidndm,
title = {{HumanoidNDM: Closing the Training-Deployment Gap of Motion Tracking via Humanoid Neural Dynamics Model}},
author = {Chao Chen and Zhenrong Wang and Jilong Wang and Yusen Feng and Jiangran Lyu and Li Yi and He Wang},
year = {2026},
eprint = {2610.34126},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34126},
}
Smooth analytic contact gradients only in the backward pass, hard-contact forward rollout unchanged: a quadruped trotting policy trains in under 2 min on a desktop and transfers to hardware without fine-tuning.只在反向传播中换用平滑的解析摩擦接触梯度，前向仍是硬接触仿真：四足小跑策略在台式机上不到 2 分钟训练完成，无需微调即可迁移到真机；在接触隐式轨迹优化中也能找到更丰富的接触解。
@misc{kim2026differentiable,
title = {{Differentiable Contact Dynamics for Legged Locomotion: Policy Learning and Contact-Implicit Trajectory Optimization}},
author = {Gijeong Kim and Seungbin Oh and Dongyun Kang and Hae-Won Park},
year = {2026},
eprint = {2610.35852},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35852},
}
Keeps a humanoid behavior foundation model frozen and trains a small lower-body residual adapter for stairs, gaps and platforms, distilled to onboard depth; tested on four trackers and on a Unitree G1 with SONIC.冻结人形行为基础模型，仅训练一个作用于下肢目标的轻量残差适配器来应对台阶、沟壑和平台，并从特权地形信息蒸馏到机载深度；在四种跟踪器上验证，并在 Unitree G1（SONIC）上真机部署。
@misc{li2026graft,
title = {{GRAFT: Geometry-Aware Residual Adaptation for Foundation Trackers}},
author = {Liang Li and Guoyao Zhang and Xingyu Chen and Sikai Wu and Qianyang Wu and Yuhang Lin and Renjing Xu and Mingliang Zhou and Diyun Xiang},
year = {2026},
eprint = {2610.36507},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36507},
}
A stochastic motion tokenizer gives a high-level generator a compact interface to a whole-body tracking policy; reports better G1 motion reconstruction than FSQ/VQ-VAE and real dance/fight/walk sequences.用随机运动分词器把行为标签或文本生成的动作压缩成离散表示，直接作为全身跟踪策略的参考；报告在 Unitree G1 动作重建上优于 FSQ 与 VQ-VAE，并在真机上执行舞蹈、格斗、行走及 35 秒的格斗转舞蹈序列。
Train the video world model's autoencoder for temporal predictability, not just reconstruction, with a past-conditioned decoder. Reports up to 5× cheaper inference, 7× faster training and lower long-horizon / OOD error.视频世界模型的潜空间不应只为重建而训练：加入时间可预测性目标并配合以历史为条件的解码器。报告推理成本最多降低 5 倍、训练快 7 倍以上，长时域与分布外误差降低 15–24%，与真实策略评测更一致。
Why action-conditioned video models ignore actions: expert demos carry too little action influence. AIM actively collects data to raise it; 500 such episodes + 3k demos match a 300k-demo model on controllability.解释动作条件视频世界模型为何忽略动作：专家示教的“动作影响力”太低，模型容量再大也无法弥补。AIM 主动采集数据提高这一指标；报告 500 条主动采集数据加 3000 条示教即可在末端可控性上追平 30 万条专家数据。
@misc{bateman2026training,
title = {{Training Controllable World Models via Action Influence Maximization}},
author = {Samuel M. Bateman and Tenny Yin and Chongyi Zheng and Lei Huang and Brian Wang and Benjamin Eysenbach and Jaime Fernández Fisac and Dhruv Shah},
year = {2026},
eprint = {2610.27387},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.27387},
}
Cross-embodiment video world model conditioned on RynnLAM's 3D latent actions, pretrained on 23 datasets (~553M frames) of human, robot and sim video; its generated data improves LIBERO and real-robot policies.以 RynnLAM 的三维潜在动作为统一条件接口的跨本体视频世界模型，在 23 个数据集（约 5.5 亿帧）的人类、机器人和仿真视频上预训练；生成的数据提升了 LIBERO 与真机策略成功率。
@misc{zhao2026rynnworld,
title = {{RynnWorld-Latent: A Cross-Embodiment World Model Conditioned on Universal Latent Actions}},
author = {Haoyu Zhao and Yinzhou Tang and Zixuan Wang and Minghao Zhu and Dongchi Huang and Xingyue Zhao and Qin Zhao and Kehan Li and Siteng Huang and Xin Li and Zhenyu Zeng and Mingxiu Chen and Deli Zhao and Zhongyu Li},
year = {2026},
eprint = {2610.36791},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36791},
}
Rewinds an action-conditioned world model to key decision points to try alternative actions, turning outcomes into action-level supervision; real data keeps correcting the model. Reports +10.7–11.9 pts on 3 real tasks.把动作条件世界模型当作反事实引擎：回到关键决策点尝试其他动作并对比结果，得到动作级监督而无需单独的价值模型；真机经验再通过 LoRA 持续修正世界模型。报告在 3 个真实任务上比最强基线高 10.7–11.9 个百分点。
@misc{luo2026rollback,
title = {{Rollback: Experience Backtracking in World Models for Policy Improvement}},
author = {Jiayi Luo and Yang Zhang and Liliang Chen and Nan Wang and Chen Gao and Boxiang Qiu and Zhuoxuan Li and Baorui Peng and Chuankang Li and Yuxiang Yan and Si Liu and Qingyun Sun and Shuicheng Yan and Maoqing Yao and Jianxin Li and Guanghui Ren},
year = {2026},
eprint = {2610.36068},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36068},
}
Shows a fixed world model's uncertainty can break when the deployment policy changes; reweights conformal calibration by a learned state-action ratio. Reports 90% sets covering 55.0% vs 90.9% on real Franka data.指出部署策略一变，固定世界模型的不确定性校准就可能失效；用域分类器学到的状态-动作密度比做加权共形校准，并给出覆盖率保证（部分用 Lean 4 形式化验证）。报告在真实 Franka 轨迹上，普通 90% 共形集合仅覆盖 55.0%，加权后为 90.9%。
@misc{mittal2026calibrated,
title = {{Calibrated for Which Policy? Conformal Uncertainty for Fixed World Models Under Intervention}},
author = {Ayush Mittal and Dhruv Gupta},
year = {2026},
eprint = {2610.35877},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2610.35877},
}
Off-route VLN agents keep following their stale history ('behavioral inertia'). MOON adds a deviation judge that clears it and a GRPO-trained recovery policy (<1% params); reports +5.2 / +6.0 points on R2R-CE / RxR-CE.发现 VLN 智能体偏离路线后会被历史记忆“惯性”拖住。MOON 给冻结的导航策略加上偏航判断器（判断何时清空历史）和 GRPO 训练的恢复策略，参数增加不到 1%；报告在 R2R-CE / RxR-CE 上分别提升 5.2 / 6.0 个百分点。
@misc{zhou2026moon,
title = {{MOON: Learning When to Forget and How to Move On in Vision-and-Language Navigation}},
author = {Jingmeng Zhou and Seson Sun and Binyao Xu and Lingming Zhang and Hanshi Wang and Fudong Ge and Weiming Hu and Jin Gao and Zhipeng Zhang},
year = {2026},
eprint = {2610.33546},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.33546},
}
Deployed robots pool their runs; failures retrieve matching successful segments from the fleet buffer to fine-tune on, with no new demos. Reports 93–99% on five real tasks; 96 robots lift phone packing 79% → 98%.部署中的机器人共享执行经验：用失败片段在机群经验库中检索相似的成功片段来微调策略，无需新增示教或人工纠正。论文报告五个真机任务成功率 93–99%，96 台机器人把手机装盒任务从 79% 提升到 98%。
@misc{yu2026learning,
title = {{Learning from Fleet Experience: Collective Experience Learning Across 96 Robots}},
author = {Chao Yu and Shu'ang Yu and Hao Lin and Hongzhi Zang and Yuxiang Yan and Chenxi Jiang and Zhen Guo and Chuankang Li and Yihao Chen and Xin Xu and Xu Fu and Jiakai Zhou and Liangzhi Shi and Hongyi Ren and Yiwei Liu and Jing Wu and Yuanqing Wang and Chen Wu and Bo Dai and Chunyu Yang and Hongliang Huang and Shuoyu Yue and Weiqi Yin and Yinuo Chen and Ruihua Chen and Chunyang Zhu and Han Xiao and Yanghao Li and Ruipu Wu and Zhikai Cai and Xiu Chen and Xu Tan and Si Liu and Yuchen Yan and Jialin Zhang and Wen Zeng and Yiming Zhou and Zixiao Huang and Xinlei Chen and Guanghui Ren and Quanlu Zhang and Boxun Li and Maoqing Yao and Yu Wang},
year = {2026},
eprint = {2610.25736},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.25736},
}
Continual off-policy RL for VLAs — small actor/critic networks carry over between tasks, distilled on a few kept episodes to avoid forgetting; reports 31–68.5% less real-robot time on new assembly tasks.面向 VLA 的持续离策略强化学习：轻量 actor/critic 在任务间延续，用少量保留的旧任务轨迹做蒸馏以防遗忘。论文报告新的接触式装配任务所需真机训练时间减少 31–68.5%，旧任务几乎不遗忘。
@misc{kim2026accrue,
title = {{ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models}},
author = {Changyeon Kim and Kimin Lee and Yashraj Narang and Yijie Guo},
year = {2026},
eprint = {2610.18596},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.18596},
}
Post-trains an open 9B robot planner (SFT + multi-turn RL) across harnesses that call VLA skills or write control code; reports LIBERO-PRO success rising from 18.1% to 71.0% without external memory.像训练编程智能体一样对 9B 开源机器人规划器做后训练（SFT + 多轮强化学习），适配调用 VLA 技能和生成控制代码两类执行框架；报告在 LIBERO-PRO 上无外部记忆成功率从 18.1% 提升到 71.0%。
@misc{liu2026agenticroborl,
title = {{AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics}},
author = {Xinxin Liu and Bingxin Xu and Wenrui Bao and Ming Li and Wentao Bao and Chen Chen and Yuzhang Shang},
year = {2026},
eprint = {2610.28589},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28589},
}
Continued pre-training study for game-playing agents: 120 runs over 0.8B–27B models, 50–1,600 h of data and multi-world mixes; reports diversity aids transfer to related worlds, little to distant ones.针对游戏中的具身智能体做持续预训练的对照研究：120 组配置覆盖 0.8B–27B 模型、50–1600 小时数据和多世界混合；报告数据多样性有助于迁移到相近世界，对差异大的世界帮助有限。
@misc{zhang2026scaling,
title = {{Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds}},
author = {Kuan Zhang and Yukun Chen and Zhihao Yang and Yue Su and Xiangnan Wu and Zirong Chen and Run Luo and Jinkun Hou and Tao Tan and Yinhe Zheng and Yiming Li},
year = {2026},
eprint = {2610.28698},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2610.28698},
}
Survey of recursive self-improvement in robot learning — improving skills, then the process that makes them — and when gains carry forward; notes no physical study has yet isolated gains from recursion itself.把"递归自我改进"放到机器人学习语境下的综述：从提升技能到改进产生技能的过程，并总结增益能持续累积的条件；指出现有真机研究尚未单独验证"递归"本身带来的收益。附论文列表。
@misc{luo2026toward,
title = {{Toward Recursive Self-Improvement in Physical AI}},
author = {Meng Luo and Zechen Bai and Shanqing Xu and Yanlin Li and Kaixin Li and Mengchen Qian and Weifeng Lu and Ran Zhang and Yingtian Zou and Ziyang Luo and Wei Chen and Hongzhan Lin},
year = {2026},
eprint = {2610.29677},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.29677},
}
PhysRSI-eval scores VLM agents on 18 manipulation tasks under a fixed trial budget on five axes, incl. learning from failure; reports similar benchmark averages hide different improvement speeds.PhysRSI-eval 在固定交互预算下用 18 个操作任务、五个维度（含从失败中改进）评测 VLM 智能体；报告公开基准平均分相近的模型改进速度差别很大，且早期物理反馈可用于预测后续进展。
@misc{li2026where,
title = {{Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy}},
author = {Zekai Li and Jiayi Yu and Jinlu Zhang and Luyang Niu and Zekun Qi and Jun Zheng and Xuchuan Chen and Dairu Liu and Yintianrun Zhang and He Wang and Li Yi},
year = {2026},
eprint = {2610.26848},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.26848},
}
Distills a pretrained VLA teacher into a policy without action pretraining via on-policy supervision plus teacher-run recoveries, triggered by a reward model (AutoDAgger); reports up to 4.1x over BC on 3 real tasks.把预训练具身基础模型（教师）蒸馏到未做动作预训练的策略：在学生访问的状态上做在线监督，并由奖励模型监控进度、自动触发教师接管采集纠错轨迹（AutoDAgger，无需人工看护）；报告在 3 个真机任务上比行为克隆最高提升 4.1 倍。
@misc{lu2026tilde,
title = {{TILDE: Teacher-in-the-Loop Distillation from Embodied Foundation Models}},
author = {Yuxiang Lu and Yantong Liu and Jinghua Hou and Xueli Sun and Ruihua Han and Zhe Liu and Hengshuang Zhao},
year = {2026},
eprint = {2610.25376},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.25376},
}
Co-trains a VLA on robot demos and noisy egocentric-video actions by weighting each source by reliability and distilling from clean future frames; reports steeper scaling than the baseline up to 30k hours of data.将机器人示教与动作标签噪声较大的第一视角视频联合训练 VLA：按数据源可靠性分配监督权重，并从干净的未来画面蒸馏表征；报告在最多 3 万小时预训练数据范围内扩展斜率高于基线。
@misc{zhang2026egoforesight,
title = {{EgoForesight: Effectively Scaling Policy Learning with Noisy Egocentric Actions}},
author = {Chubin Zhang and Changyuan Wang and Yirui Wang and Ao Li and Shilin Ma and Yue Su and Cai Zhou and Xiuwei Xu and Tianle Zhang and Yansong Tang},
year = {2026},
eprint = {2610.31845},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31845},
}
TIDE updates a pretrained policy online from human interventions and its own rollouts, with no critic or reward model; reports gains on four real bimanual tasks, each session finishing in under an hour.TIDE 在部署过程中利用人类干预和策略自身的执行数据在线更新预训练策略，无需价值函数或奖励模型；报告在四个真实双臂任务上优于初始策略，每次在线后训练不到一小时完成。
@misc{gu2026training,
title = {{Training in Deployment: Human-in-the-loop Online Post-Training for Robot Manipulation}},
author = {Chun Gu and Xiaofei Wei and Yiming Chen and Bozhou Zhang and Juntong Zhang and Xiatian Zhu and Li Zhang},
year = {2026},
eprint = {2610.29264},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.29264},
}
Uses RL as a demo generator, not a controller: a latent-conditioned policy finds several distinct successful solutions, exported as an imitation dataset; reports 78% real-robot success vs. 39% for the best baseline.把强化学习当作示教数据生成器而非部署控制器：以隐变量为条件的策略为同一任务找到多种不同的成功解法，导出为模仿学习数据集；报告真机下游成功率从最强基线的 39% 提升到 78%。
@misc{park2026discodemo,
title = {{DiscoDemo: Discovering Efficient and Diverse Robot Demonstrations for Imitation Learning}},
author = {Minho Park and Kinam Kim and Donghu Kim and Byungkun Lee and Dongyoon Hwang and Yongjae Shin and Junha Hyung and Hojoon Lee and Jaegul Choo},
year = {2026},
eprint = {2610.32093},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.32093},
}
Replay-free lifelong imitation learning: hierarchical dual-routing attention retrieves experts by instruction, then refines by context, plus key regularization against drift. Reports beating replay baselines on LIBERO.无回放的终身模仿学习：分层双路由注意力先按指令检索专家、再按上下文细化，并用键级几何正则抑制表征漂移带来的路由干扰；报告在多个 LIBERO 任务序列上优于无回放和有回放基线。
Fine-tunes diffusion and flow policies with automatic preference labels: rollouts are ranked by progress measured against a few expert demos, no human labels or reward design. Reports beating SFT on long-horizon tasks.用自动生成的偏好标签微调扩散与流匹配策略：以少量专家示教为参照衡量自主回放的顺序进度并据此排序，无需人工标注或手工奖励；报告在长时程操作任务上优于监督微调基线。
@misc{luu2026progress,
title = {{Progress-Guided Preference Generation for Aligning Generative Control Policies}},
author = {Tung M. Luu and Tri Ton and Duc M. Nguyen and Xuan Thanh Nguyen and Nguyen Gia Binh and Quang Tan Nguyen and Vu Duong and Daniel Sonntag and Duy M. H. Nguyen and Vien Anh Ngo and Chang D. Yoo},
year = {2026},
eprint = {2610.25692},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2610.25692},
}
Tests robot value (progress) models on 15 sim and 20 real tasks across task state, progress, failure/recovery and consistency; reports that good forward-progress scores hide weak history and recovery reasoning.在 15 个仿真和 20 个真实任务上评测机器人价值（进度）模型，覆盖任务状态、进度监测、失败与恢复、一致性四个维度；报告称前向进度得分高的模型在依赖历史和恢复判断上仍然薄弱。
@misc{liu2026robovalue,
title = {{RoboValue: A Fine-Grained Sim-and-Real Benchmark for Unified Evaluation of Robotic Value Models}},
author = {Shengbang Liu and Zhengye Du and Zhilong Wan and Honghao Su and Chenxiang Xia and Chang Ge and Jinyang Xiao and Nan Wang and Zhipeng Hu and Huining Yuan and Shu'ang Yu and Yue Chen and Tianxing Chen and Chuankang Li and Maoqing Yao and Wenbo Ding and Yu Wang and Guanghui Ren and Chao Yu},
year = {2026},
eprint = {2610.26663},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.26663},
}
The open data engine behind GR00T training: models that write timestamped action captions, dense process rewards and episode outcome rewards for human and robot video; reports SOTA on WGO-Bench, RoboReward and Robometer.GR00T 训练所用的开放数据引擎：一组模型为人类与机器人视频生成带时间戳的动作描述、稠密过程奖励和整段结果奖励；报告在 WGO-Bench、RoboReward 与 Robometer 上达到最优。
@misc{leng2026flora,
title = {{FLORA: An Open Model Suite for Scaling Robotics Data Engines}},
author = {Sicong Leng and Yunze Man and Zhiqi Li and Jing Wang and Yuhe Zhang and Ryan Punamiya and Siddharth Patki and Kaiyuan Zheng and Zi-ang Cao and Zhiding Yu and Sifei Liu and Danfei Xu and Ben Levin and Jan Kautz and Yuke Zhu and Johan Bjorck and Linxi "Jim" Fan},
year = {2026},
eprint = {2610.31779},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31779},
}
Learns task-centric latent actions from videos (in DINO feature space, filtered by language) so one policy can be pretrained across robots and human video; reports beating OpenVLA with far less pretraining compute.在 DINO 特征空间中、借助语言指令过滤，从视频里学习“任务相关”的隐动作，使策略可跨机器人与人类视频预训练；报告以少得多的预训练算力超过 OpenVLA。
@misc{bu2025univla,
title = {{UniVLA: Learning to Act Anywhere with Task-centric Latent Actions}},
author = {Qingwen Bu and Yanting Yang and Jisong Cai and Shenyuan Gao and Guanghui Ren and Maoqing Yao and Ping Luo and Hongyang Li},
year = {2025},
eprint = {2505.06111},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2505.06111},
}
A slow latent diffusion policy plans action chunks while a fast policy reacts to tactile/force feedback; reports >35% higher scores than visual imitation baselines on three contact-rich tasks.慢速扩散策略在隐空间规划动作块，快速策略根据触觉/力反馈高频调整；论文报告在三个接触丰富任务上比纯视觉模仿学习基线总分高 35% 以上。
Fine-tunes video world models on a robot to dream new demos, labels them with latent actions or an inverse dynamics model, and trains policies on them; a humanoid learns 20+ new behaviors from one-scene teleop data.把视频世界模型微调到目标机器人上“做梦”生成新示范，再用隐动作或逆动力学模型补上动作标签用于训练策略；人形机器人仅凭单一场景的遥操作数据学会 20 多种新行为。
@misc{jang2025dreamgen,
title = {{DreamGen: Unlocking Generalization in Robot Learning through Video World Models}},
author = {Joel Jang and Seonghyeon Ye and Zongyu Lin and Jiannan Xiang and Johan Bjorck and Yu Fang and Fengyuan Hu and Spencer Huang and Kaushil Kundalia and Yen-Chen Lin and Loic Magne and Ajay Mandlekar and Avnish Narayan and You Liang Tan and Guanzhi Wang and Jing Wang and Qi Wang and Yinzhen Xu and Xiaohui Zeng and Kaiyuan Zheng and Ruijie Zheng and Ming-Yu Liu and Luke Zettlemoyer and Dieter Fox and Jan Kautz and Scott Reed and Yuke Zhu and Linxi Fan},
year = {2025},
eprint = {2505.12705},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2505.12705},
}
DSRL — adapts a frozen diffusion policy by running RL over its input noise, not its weights; needs only forward passes, works with π0, and reports real-robot success rising from 20% to 90% in under 50 online episodes.DSRL：不改扩散策略的权重，而是对其输入的初始噪声做强化学习来"引导"输出动作；只需前向推理、可用于 π0，论文报告真机任务在不到 50 次在线交互内成功率从 20% 升至 90%。
@misc{wagenmaker2025steering,
title = {{Steering Your Diffusion Policy with Latent Space Reinforcement Learning}},
author = {Andrew Wagenmaker and Mitsuhiko Nakamoto and Yunchu Zhang and Seohong Park and Waleed Yagoub and Anusha Nagabandi and Abhishek Gupta and Sergey Levine},
year = {2025},
eprint = {2506.15799},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2506.15799},
}
FQL — a flow-matching BC policy plus a one-step policy that maximizes Q while distilling from it; no backprop through flow steps, strong on 73 OGBench / D4RL offline and offline-to-online tasks.FQL：先用行为克隆训练流匹配策略，再训练一个最大化 Q 值、同时向流策略蒸馏的单步策略；避免对多步流采样反向传播，在 73 个 OGBench / D4RL 离线及离线转在线任务上表现突出。
Online RL for VLAs with GRPO and only binary success rewards on OpenVLA-OFT, plus exploration tweaks; reports 99.1% on LIBERO and 70.4% on RoboTwin 1.0 (vs 39.8% for the SFT baseline).在 OpenVLA-OFT 上用 GRPO 与仅"成功/失败"二值奖励做在线强化学习，并加入若干促进探索的改动；论文报告 LIBERO 成功率 99.1%、RoboTwin 1.0 为 70.4%（SFT 基线 39.8%）。
@misc{li2025simplevla,
title = {{SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning}},
author = {Haozhan Li and Yuxin Zuo and Jiale Yu and Yuhao Zhang and Zhaohui Yang and Kaiyan Zhang and Xuekai Zhu and Yuchen Zhang and Tianxing Chen and Ganqu Cui and Dehui Wang and Dingxiang Luo and Yuchen Fan and Youbang Sun and Jia Zeng and Jiangmiao Pang and Shanghang Zhang and Yu Wang and Yao Mu and Bowen Zhou and Ning Ding},
year = {2025},
eprint = {2509.09674},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2509.09674},
}
Open RL training framework for VLAs (PPO/GRPO across GPU-parallel simulators); reports 98.11% on 130 LIBERO tasks and 97.66% on 25 ManiSkill tasks after RL — a common base for VLA+RL work.面向 VLA 的开源强化学习训练框架（PPO/GRPO，支持 GPU 并行仿真器）；论文报告经 RL 后在 130 个 LIBERO 任务上成功率 98.11%、25 个 ManiSkill 任务上 97.66%，是 VLA+RL 研究常用的底座。
@misc{zang2025rlinf,
title = {{RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models}},
author = {Hongzhi Zang and Mingjie Wei and Si Xu and Yongji Wu and Zhen Guo and Yuanqing Wang and Hao Lin and Peihong Wang and Liangzhi Shi and Yuqing Xie and Zhexuan Xu and Zhihao Liu and Kang Chen and Wenhao Tang and Quanlu Zhang and Weinan Zhang and Chao Yu and Yu Wang},
year = {2025},
eprint = {2510.06710},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2510.06710},
}
Exoskeleton demos are converted into pseudo-robot demonstrations; the paper reports its RISE-2 policy trained on them alone, with no robot data, performs comparably to one trained on teleoperated robot data.将外骨骼示教转换为“伪机器人”示教；论文报告仅用这些数据（无任何机器人数据）训练的 RISE-2 策略，效果与用遥操作机器人数据训练的策略相当。
@misc{fang2025airexo,
title = {{AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons}},
author = {Hongjie Fang and Chenxi Wang and Yiming Wang and Jingjing Chen and Shangning Xia and Jun Lv and Zihao He and Xiyan Yi and Yunhan Guo and Xinyu Zhan and Lixin Yang and Weiming Wang and Cewu Lu and Hao-Shu Fang},
year = {2025},
eprint = {2503.03081},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2503.03081},
}
HPT: per-robot stems turn proprioception and vision into a few tokens for one shared transformer trunk, pretrained on 52 datasets (robot, sim, human video); reports 20%+ gains on unseen tasks.HPT：为每种机器人设计“输入头”，把本体感知与视觉映射成短 token 序列，送入共享的 Transformer 主干；在 52 个数据集（真机、仿真、人类视频）上预训练，报告在未见任务上微调提升 20% 以上。
LAPA pretrains a VLA without action labels: it learns discrete latent actions between video frames, predicts them from image + text, then maps them to robot actions; reports beating OpenVLA on real tasks.LAPA 无需动作标注即可预训练 VLA：先从视频相邻帧间学习离散隐动作，再训练模型由图像和指令预测隐动作，最后用少量机器人数据映射到真实动作；报告在真机任务上优于 OpenVLA。
@misc{ye2024latent,
title = {{Latent Action Pretraining from Videos}},
author = {Seonghyeon Ye and Joel Jang and Byeongguk Jeon and Sejune Joo and Jianwei Yang and Baolin Peng and Ajay Mandlekar and Reuben Tan and Yu-Wei Chao and Bill Yuchen Lin and Lars Liden and Kimin Lee and Jianfeng Gao and Luke Zettlemoyer and Dieter Fox and Minjoon Seo},
year = {2024},
eprint = {2410.11758},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.11758},
}
Treats egocentric human video with 3D hand tracking (Project Aria glasses) as robot data and co-trains one policy on both; reports an extra hour of human hand data helps more than an extra hour of robot data.把佩戴 Project Aria 眼镜采集、带三维手部跟踪的第一视角人类视频当作机器人数据，与机器人数据共同训练同一个策略；论文报告多 1 小时人手数据比多 1 小时机器人数据更有用。
@misc{kareer2024egomimic,
title = {{EgoMimic: Scaling Imitation Learning via Egocentric Video}},
author = {Simar Kareer and Dhruv Patel and Ryan Punamiya and Pranay Mathur and Shuo Cheng and Chen Wang and Judy Hoffman and Danfei Xu},
year = {2024},
eprint = {2410.24221},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.24221},
}
Unitree's official starting point for RL locomotion on Go2, H1 and G1 — train in Isaac Gym, check sim-to-sim in MuJoCo, then deploy to the real robot.宇树官方的强化学习运动控制入门仓库，支持 Go2、H1、G1：在 Isaac Gym 中训练，在 MuJoCo 中做 sim2sim 验证，再部署到真机。
An open-source stack for training manipulation policies with RL on a real arm — learned reward classifiers, reset-free training and an impedance controller.在真实机械臂上用强化学习训练操作策略的开源软件栈：学习式奖励分类器、免重置训练与阻抗控制器。
@misc{luo2024serl,
title = {{SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning}},
author = {Jianlan Luo and Zheyuan Hu and Charles Xu and You Liang Tan and Jacob Berg and Archit Sharma and Stefan Schaal and Chelsea Finn and Abhishek Gupta and Sergey Levine},
year = {2024},
eprint = {2401.16013},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2401.16013},
}
DPPO — fine-tunes a pre-trained diffusion policy with PPO by treating the denoising steps as part of the MDP; reports stable gains over other RL fine-tuning methods and sim-to-real transfer.DPPO：把去噪步骤视为 MDP 的一部分，用 PPO 微调预训练扩散策略；报告称比其他强化学习微调方法更稳定，并实现了仿真到真机迁移。
@misc{ren2024diffusion,
title = {{Diffusion Policy Policy Optimization}},
author = {Allen Z. Ren and Justin Lidard and Lars L. Ankile and Anthony Simeonov and Pulkit Agrawal and Anirudha Majumdar and Benjamin Burchfiel and Hongkai Dai and Max Simchowitz},
year = {2024},
eprint = {2409.00588},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2409.00588},
}
PIRLNav — behavior cloning on human ObjectNav demonstrations, then RL fine-tuning of the same policy; the paper reports 65.0% success on Habitat ObjectNav, 5 points above the previous best.PIRLNav：先在人类 ObjectNav 演示上做行为克隆，再用强化学习微调同一策略；论文报告在 Habitat ObjectNav 上成功率 65.0%，比此前最佳高 5 个百分点。
@misc{ramrakhya2023pirlnav,
title = {{PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNav}},
author = {Ram Ramrakhya and Dhruv Batra and Erik Wijmans and Abhishek Das},
year = {2023},
eprint = {2301.07302},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2301.07302},
}
RLPD — a few simple changes to off-policy RL (symmetric sampling of offline and online data, layer norm, ensembles) make prior data speed up online learning.RLPD：对离策略强化学习做几处简单改动（离线/在线数据对称采样、层归一化、集成），让已有数据加速在线学习。
@misc{ball2023efficient,
title = {{Efficient Online Reinforcement Learning with Offline Data}},
author = {Philip J. Ball and Laura Smith and Ilya Kostrikov and Sergey Levine},
year = {2023},
eprint = {2302.02948},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2302.02948},
}
Keeps conservative offline Q-values from dropping below a reference policy's value, which avoids the early performance dip when an offline-pretrained policy starts online fine-tuning.约束保守的离线 Q 值不低于参考策略的价值，从而避免离线预训练策略开始在线微调时常见的性能先降后升。
@misc{nakamoto2023cal,
title = {{Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning}},
author = {Mitsuhiko Nakamoto and Yuexiang Zhai and Anikait Singh and Max Sobol Mark and Yi Ma and Chelsea Finn and Aviral Kumar and Sergey Levine},
year = {2023},
eprint = {2303.05479},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2303.05479},
}
Discretizes each action dimension and predicts Q-values per dimension autoregressively with a Transformer, plus a conservative regularizer; trained offline on large real robot datasets mixing demos and autonomous data.将每个动作维度离散化，用 Transformer 自回归地逐维预测 Q 值，并加入保守正则；在混合演示与自主采集数据的大规模真实机器人数据集上离线训练。
@misc{chebotar2023q,
title = {{Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions}},
author = {Yevgen Chebotar and Quan Vuong and Alex Irpan and Karol Hausman and Fei Xia and Yao Lu and Aviral Kumar and Tianhe Yu and Alexander Herzog and Karl Pertsch and Keerthana Gopalakrishnan and Julian Ibarz and Ofir Nachum and Sumedh Sontakke and Grecia Salazar and Huong T Tran and Jodilyn Peralta and Clayton Tan and Deeksha Manjunath and Jaspiar Singht and Brianna Zitkovich and Tomas Jackson and Kanishka Rao and Chelsea Finn and Sergey Levine},
year = {2023},
eprint = {2309.10150},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2309.10150},
}
Open-source tactile RL environments with sim-to-real transfer across three low-cost optical sensors (TacTip, DIGIT, DigiTac), for comparing them on the same tasks.开源触觉强化学习环境，支持三种低成本光学触觉传感器（TacTip、DIGIT、DigiTac）的仿真到真机迁移，便于在相同任务上比较。
@misc{lin2022tactile,
title = {{Tactile Gym 2.0: Sim-to-real Deep Reinforcement Learning for Comparing Low-cost High-Resolution Robot Touch}},
author = {Yijiong Lin and John Lloyd and Alex Church and Nathan F. Lepora},
year = {2022},
eprint = {2207.10763},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2207.10763},
}
BeT — clusters actions into k bins plus a continuous offset so a transformer can imitate multimodal demonstrations instead of averaging them.BeT：把动作聚成 k 个类别再加连续偏移，让 Transformer 能模仿多模态示教，而不是把不同模式平均掉。
@misc{shafiullah2022behavior,
title = {{Behavior Transformers: Cloning $k$ modes with one stone}},
author = {Nur Muhammad Mahi Shafiullah and Zichen Jeff Cui and Ariuntuya Altanzaya and Lerrel Pinto},
year = {2022},
eprint = {2206.11251},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2206.11251},
}
A discriminator trained on unstructured mocap clips acts as a style reward, so RL learns tasks with natural motion without hand-picked reference clips; widely reused as a gait prior for legged robots.用在无结构动捕片段上训练的判别器作为"风格奖励"，强化学习无需逐段指定参考动作即可学到自然的动作；后来被广泛用作足式机器人的步态先验。
@misc{peng2021amp,
title = {{AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control}},
author = {Xue Bin Peng and Ze Ma and Pieter Abbeel and Sergey Levine and Angjoo Kanazawa},
year = {2021},
eprint = {2104.02180},
archivePrefix = {arXiv},
primaryClass = {cs.GR},
url = {https://arxiv.org/abs/2104.02180},
}
robomimic — a controlled study of imitation and offline RL on human-collected manipulation demos (demo quality, observation choices, history-aware BC), released with the framework and datasets.robomimic：在人类采集的操作示教数据上系统比较模仿学习与离线强化学习（示教质量、观测选择、带历史的行为克隆），并开源框架与数据集。
@misc{mandlekar2021what,
title = {{What Matters in Learning from Offline Human Demonstrations for Robot Manipulation}},
author = {Ajay Mandlekar and Danfei Xu and Josiah Wong and Soroush Nasiriany and Chen Wang and Rohun Kulkarni and Li Fei-Fei and Silvio Savarese and Yuke Zhu and Roberto Martín-Martín},
year = {2021},
eprint = {2108.03298},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2108.03298},
}
IQL — learns values with expectile regression without ever querying actions outside the dataset, then extracts a policy by advantage-weighted regression; a common base for offline-to-online robot RL.IQL：用期望分位数回归学习价值函数、从不查询数据集外的动作，再用优势加权回归提取策略；是机器人离线到在线强化学习的常用基础。
Recasts RL as sequence modeling — a GPT-style model predicts actions conditioned on the desired return; an early bridge between transformers and control.把强化学习改写成序列建模：GPT 式模型以期望回报为条件预测动作；是 Transformer 与控制结合的早期代表。
@misc{chen2021decision,
title = {{Decision Transformer: Reinforcement Learning via Sequence Modeling}},
author = {Lili Chen and Kevin Lu and Aravind Rajeswaran and Kimin Lee and Aditya Grover and Michael Laskin and Pieter Abbeel and Aravind Srinivas and Igor Mordatch},
year = {2021},
eprint = {2106.01345},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2106.01345},
}
Represents the policy as an energy-based model over actions instead of a regressor, which handles multimodal and discontinuous demonstrations much better.用基于能量的模型（而非直接回归）表示策略，更好地处理多模态、不连续的示范数据。
@misc{florence2021implicit,
title = {{Implicit Behavioral Cloning}},
author = {Pete Florence and Corey Lynch and Andy Zeng and Oscar Ramirez and Ayzaan Wahid and Laura Downs and Adrian Wong and Johnny Lee and Igor Mordatch and Jonathan Tompson},
year = {2021},
eprint = {2109.00137},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2109.00137},
}
DrQ-v2 - a simple model-free actor-critic with image augmentation that learns continuous control from pixels (incl. humanoid tasks in the DeepMind Control Suite) on a single GPU; a common visual-RL baseline.DrQ-v2：带图像增强的简单无模型 actor-critic，单 GPU 即可从像素学习连续控制（包括 DeepMind Control Suite 的人形任务），是常用的视觉强化学习基线。
JAX physics engine that runs thousands of environments in parallel on GPU/TPU together with the RL training loop, cutting locomotion training from hours to minutes.基于 JAX 的物理引擎，在 GPU/TPU 上与强化学习训练循环一起并行运行数千个环境，把运动控制训练从数小时缩短到数分钟。
@misc{freeman2021brax,
title = {{Brax: A Differentiable Physics Engine for Large Scale Rigid Body Simulation}},
author = {C. Daniel Freeman and Erik Frey and Anton Raichuk and Sertan Girgin and Igor Mordatch and Olivier Bachem},
year = {2021},
eprint = {2106.13281},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2106.13281},
}
CQL — penalizes Q-values on actions outside the dataset so the learned value is a lower bound; a standard offline-RL baseline for learning from fixed robot logs.CQL：压低数据集外动作的 Q 值，使学到的价值成为下界；是从固定机器人日志中学习的标准离线强化学习基线。
Advantage-weighted actor-critic that pre-trains on prior demonstrations and keeps improving online, shown on real dexterous-hand tasks.优势加权的 Actor-Critic：先用已有示范数据预训练，再在线持续提升，并在真实灵巧手任务上验证。
DeepMind's Python stack on top of MuJoCo — bindings, PyMJCF and Composer for building models and tasks, the standard Control Suite benchmarks, plus locomotion and arm-and-bricks manipulation tasks.DeepMind 基于 MuJoCo 的 Python 工具栈：底层绑定、用于程序化建模与编写任务的 PyMJCF 和 Composer、作为标准基准的 Control Suite，以及运动控制和机械臂拼积木操作任务。
@misc{tassa2020dm,
title = {{dm_control: Software and Tasks for Continuous Control}},
author = {Yuval Tassa and Saran Tunyasuvunakool and Alistair Muldal and Yotam Doron and Piotr Trochim and Siqi Liu and Steven Bohez and Josh Merel and Tom Erez and Timothy Lillicrap and Nicolas Heess},
year = {2020},
eprint = {2006.12983},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2006.12983},
}
An early shared dataset of robot interaction videos from seven robot platforms, used to pretrain visual dynamics models that transfer to new robots - a forerunner of Open X-Embodiment.早期的共享机器人交互视频数据集，涵盖七种机器人平台，用于预训练可迁移到新机器人的视觉动力学模型，是 Open X-Embodiment 的先驱。
@misc{dasari2019robonet,
title = {{RoboNet: Large-Scale Multi-Robot Learning}},
author = {Sudeep Dasari and Frederik Ebert and Stephen Tian and Suraj Nair and Bernadette Bucher and Karl Schmeckpeper and Siddharth Singh and Sergey Levine and Chelsea Finn},
year = {2019},
eprint = {1910.11215},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/1910.11215},
}
Learns a compact vision + force/torque representation with self-supervised objectives, then trains a peg-insertion policy with RL on top of it on a real robot.用自监督目标学习视觉 + 力/力矩的紧凑联合表征，再在其上用强化学习在真实机器人上训练插孔策略。
@misc{lee2018making,
title = {{Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks}},
author = {Michelle A. Lee and Yuke Zhu and Krishnan Srinivasan and Parth Shah and Silvio Savarese and Li Fei-Fei and Animesh Garg and Jeannette Bohg},
year = {2018},
eprint = {1810.10191},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/1810.10191},
}
Off-policy RL with an entropy bonus; sample-efficient enough to become the default for learning directly on real robots.带熵奖励的离策略强化学习；样本效率较高，成为真机上直接学习的常用默认算法。
@misc{haarnoja2018soft,
title = {{Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor}},
author = {Tuomas Haarnoja and Aurick Zhou and Pieter Abbeel and Sergey Levine},
year = {2018},
eprint = {1801.01290},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/1801.01290},
}
TD3 — fixes DDPG's value overestimation with twin critics, delayed policy updates and target smoothing; still a standard continuous-control baseline.TD3：通过双 Critic、延迟策略更新和目标策略平滑解决 DDPG 的价值高估问题，至今仍是连续控制的常用基线。
@misc{fujimoto2018addressing,
title = {{Addressing Function Approximation Error in Actor-Critic Methods}},
author = {Scott Fujimoto and Herke van Hoof and David Meger},
year = {2018},
eprint = {1802.09477},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/1802.09477},
}
Off-policy Q-learning with a cross-entropy-method action optimizer, trained on several hundred thousand real grasp attempts collected by a fleet of robots; reports 96% grasp success on unseen objects.使用交叉熵方法优化动作的离策略 Q 学习，在机器人集群采集的数十万次真实抓取上训练；报告对未见物体的抓取成功率为 96%。
@misc{kalashnikov2018qt,
title = {{QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation}},
author = {Dmitry Kalashnikov and Alex Irpan and Peter Pastor and Julian Ibarz and Alexander Herzog and Eric Jang and Deirdre Quillen and Ethan Holly and Mrinal Kalakrishnan and Vincent Vanhoucke and Sergey Levine},
year = {2018},
eprint = {1806.10293},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/1806.10293},
}
GAIL — learns a policy whose state-action distribution a discriminator cannot tell apart from the expert's, skipping explicit reward recovery.GAIL：训练策略使判别器无法区分其状态-动作分布与专家数据，无需显式恢复奖励函数。
DAgger — query the expert on the states the learner itself visits, fixing the compounding errors of plain behavior cloning.DAgger：在学习者自己访问到的状态上向专家查询标注，解决单纯行为克隆的误差累积问题。
@misc{ross2010reduction,
title = {{A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning}},
author = {Stephane Ross and Geoffrey J. Gordon and J. Andrew Bagnell},
year = {2010},
eprint = {1011.0686},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/1011.0686},
}
Chaehyun Kim, Chi-Pin Huang, Jisoo Kim, Min-Hung Chen et al.
Abstract摘要
Vision-Language-Action (VLA) models have made substantial progress in robotic manipulation, but their generalization remains constrained by the diversity and coverage of action-labeled robot demonstrations. Human videos provide much broader behavioral and environmental experience, while learning from them faces a fundamental trade-off between scalable data and explicit action grounding. We introduce ThinkWorld, a VLA learning framework that grounds action reasoning in world dynamics and strengthens it through reinforcement learning with world imagination. ThinkWorld represents each action …
@misc{kim2026thinkworld,
title = {{ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning}},
author = {Chaehyun Kim and Chi-Pin Huang and Jisoo Kim and Min-Hung Chen and Yu-Chiang Frank Wang and Seungryong Kim and Fu-En Yang},
year = {2026},
eprint = {2610.37666},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37666},
}
Anukriti Singh, Maxime Caniot, Pratap Tokekar, Firas Abi-Farraj
Abstract摘要
Imitation policies that reliably execute most of a manipulation trajectory can still fail in localized regions requiring last-mile precision. Updating the full policy or exploring residual actions everywhere treats this local failure as a global learning problem. We introduce Adaptive Residual Correction (ARC), a lightweight post-training module that uses reinforcement learning (RL) to learn a residual correction on top of a frozen policy while using sparse human corrections to structure autonomous exploration. Rather than treating human corrections as imitation targets, ARC uses …
@misc{singh2026when,
title = {{When, Where, and What to Correct: Human-Guided Adaptive Residual Correction for Robot Policies}},
author = {Anukriti Singh and Maxime Caniot and Pratap Tokekar and Firas Abi-Farraj},
year = {2026},
eprint = {2610.37547},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37547},
}
Dequan Ou, Austin Lopez, Quentin Missinne, Christophe De Wagter et al.
Abstract摘要
Event cameras offer low latency, high dynamic range, and milliwatt-level power consumption, making them ideal for agile robots such as tiny drones navigating through cluttered environments. However, existing simulators generally render photorealistic images before converting them into events, making large-scale reinforcement learning prohibitively slow. We propose an analytic event simulator that exploits what event cameras fundamentally measure - moving contrast edges - to synthesize event streams directly from scene geometry and ego-motion without visual rendering. Our simulator generates a …
@misc{ou2026rendering,
title = {{Rendering-Free Event Frame Simulation for Fast Reinforcement Learning of Autonomous Robot Navigation}},
author = {Dequan Ou and Austin Lopez and Quentin Missinne and Christophe De Wagter and Guido C. H. E. de Croon},
year = {2026},
eprint = {2610.37543},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37543},
}
A joint of pipe run into the hole swings from the elevator over the well and strikes a fixed stabbing guide at whatever speed the swing has. StabBot, a robotic stabbing system, predicts the swing and meets the pipe at its own speed. A pendulum Kalman filter tuned from the pipe tally tracks the joint in stereo images. A six-axis arm with an open V-shaped roller head closes in at matched velocity, confirms the last millimetres by touch, aims the pin over the box when the elevator hangs off centre, and follows the block down with lateral compliance. A ResNet-18 keypoint network locates the pipe, …
Medium altitude, long endurance unmanned aerial systems such as Shahed-type strike drones are an unanswered threat in modern conflicts. Conventional air defense systems are too expensive to provide a feasible large-scale solution. Quadcopter interceptors are a cheap, battle-proven alternative, but their extremely limited flight time restricts them to low altitude targets at short range, whereas fixed-wing platforms have the endurance to reach higher altitude targets from longer distances. However, such interceptors are still operated manually. Recent developments in learning-based controllers …
Yifei Yuan, Jakob Wolf, Ghaith Androwis, Xianlian Zhou
Abstract摘要
Objective: This study aims to determine whether a hip-exoskeleton policy trained in simulation, by co-adapting with a musculoskeletal human model under a metabolic-energy reward, reduces the metabolic cost of walking in new users without per-user tuning, and how mechanical-power shaping affects the learned assistance. Methods: We developed MARL-Exo, a multi-agent reinforcement learning framework in which a full-body human model (304 muscle actuators; randomized anatomy, gait, and speed) and a bilateral hip-exoskeleton policy are trained concurrently in two stages. The exoskeleton observes …
@misc{yuan2026multi,
title = {{Multi-Agent Reinforcement Learning of Co-Adaptive Hip Exoskeleton Control (MARL-Exo) for Metabolic Energy Reduction}},
author = {Yifei Yuan and Jakob Wolf and Ghaith Androwis and Xianlian Zhou},
year = {2026},
eprint = {2610.37441},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37441},
}
Sinjae Kang, Dong Kyu Shin, Jaehong Kim, Kamin Lee et al.
Abstract摘要
Robot policy evaluation often relies on a binary success signal for task completion. In real-world deployment, however, desired execution behaviors are also defined by quality attributes such as execution speed and motion smoothness, which need to be dynamically controlled according to task requirements. To address this, we introduce Quality-Controllable Policy (QCP), a robot policy that enables inference-time control over both individual quality attributes and their combinations. To extend the controllable quality range beyond initial demonstrations, we further propose an iterative hindsight …
@misc{kang2026learning,
title = {{Learning Quality-Controllable Robot Policies with Iterative Hindsight Relabeling}},
author = {Sinjae Kang and Dong Kyu Shin and Jaehong Kim and Kamin Lee and Hyoeun Kim and Kaixin Wang and Li Zhao and Seonghyeon Ye and others},
year = {2026},
eprint = {2610.37393},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37393},
}
Bojun Zou, Wenhao Yu, Jiaohao Li, Jiaxuan Lin et al.
Abstract摘要
Vision-Language-Action (VLA) models integrate high-level reasoning with low-level action execution for end-to-end robotic control. However, most VLA models target static manipulation in unconstrained environments, overlooking spatial constraints and dynamic obstacles. Fine-tuning VLAs on obstacle-avoidance data is a straightforward solution, but collecting such data is costly, especially for dynamic scenarios, and may degrade their original task capabilities. To address these limitations, we propose VLA-Stem, a lightweight, reinforcement-learning-based safety mechanism for VLA systems. …
@misc{zou2026vla,
title = {{VLA-Stem: A Lightweight Safety Mechanism for Vision-Language-Action Models via Reinforcement Learning}},
author = {Bojun Zou and Wenhao Yu and Jiaohao Li and Jiaxuan Lin and Guanting Ye and Haohan Geng and Weihao Yu and Yanyong Zhang and others},
year = {2026},
eprint = {2610.37378},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37378},
}
Leon Magnus, Nico Bohlinger, Jan Peters, Oleg Arenz
Abstract摘要
Humanoids and other legged robots operating in complex environments must remember information that leaves their field of view and only becomes relevant much later. Reinforcement learning policies address this using memory networks such as LSTMs that are trained end-to-end with the policy to memorize such task-relevant aspects. Yet, such architectures often fail to preserve sufficient information over long horizons. In this work, we propose a network architecture based on Hadamard memory that specifically targets robot navigation. Our Local Spatial Matrix Memory (LSMM) enables RL agents to …
@misc{magnus2026remember,
title = {{Remember Where You Walk: Learning Spatial Memory for End-to-End Legged Robot Exploration}},
author = {Leon Magnus and Nico Bohlinger and Jan Peters and Oleg Arenz},
year = {2026},
eprint = {2610.37203},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37203},
}
We trained four widely used imitation-learning policies, ACT, Diffusion Policy, SmolVLA and $\pi_{0.5}$, on nested subsets of 12, 25, 50, 100 and 200 teleoperated demonstrations of one pick-and-place task with distractor objects, using one fixed training recipe per policy. Every trained model was then run on a physical SO-101 arm on a fixed set of object placements: 1,340 scored episodes in 126 blocks. ACT, trained from scratch, rose from 1 success in 40 episodes at 12 demonstrations to 37 in 40 at 100. $\pi_{0.5}$ with its action expert fine-tuned was the best policy when data was scarce (16 …
Yang Liu, Xiaoli Liu, Liuzhao Li, Yuduo Song et al.
Abstract摘要
World-action models (WAMs) incorporate the prediction of future observations into robot policy learning. However, most WAMs build on billion-parameter vision-language or video-generation backbones, which incur high training cost and inference latency. In this letter, we present GAWM (Geometry-Aware World Model), a compact multi-task WAM that predicts future observations in the latent space of a frozen DINOv3 encoder. GAWM compresses the features of each camera view into a small set of tokens. A lightweight predictor, conditioned on the task rather than on actions, forecasts these tokens at …
@misc{liu2026gawm,
title = {{GAWM: A Compact Latent World-Action Model for Multi-Task Robot Manipulation}},
author = {Yang Liu and Xiaoli Liu and Liuzhao Li and Yuduo Song and Muyuan Liu and Xiang Gao},
year = {2026},
eprint = {2610.37091},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37091},
}
Zichen Yan, Rui Huang, Shenao Wang, Yucong Wang et al.
Abstract摘要
Image-goal navigation (ImageNav) requires an agent to reach a location specified by a goal image in an unknown environment. For drones, this task demands exploration, goal- directed navigation, and collision avoidance under limited onboard computing resources. Existing aerial ImageNav methods are built on single-view observations, requiring additional rotation and exploration to search for goal-relevant cues outside the field of view. In this paper, we present Omni-IGN, a fully onboard ImageNav system that integrates quad-fisheye perception, an attention-based reinforcement learning (RL) …
@misc{yan2026omni,
title = {{Omni-IGN: Onboard Image-Goal Navigation for Drones with Omnidirectional Vision}},
author = {Zichen Yan and Rui Huang and Shenao Wang and Yucong Wang and Nuthasith Gerdpratoom and Lin Zhao},
year = {2026},
eprint = {2610.36968},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36968},
}
Marc Ziegler, Batu \"Ozen, Sami Mentes, Thomas Waschulzik et al.
Abstract摘要
Robot foundation models may generalize across tasks and embodiments, but their actions still depend on sensors that can fail. When the policy is frozen or inaccessible, can an external monitor diagnose and repair corrupted inputs without access to its internals? We test a structured surrogate that separates periodic gait from sensor-dependent corrections to guide input repair around a frozen locomotion policy. On simulated Go2, offline diagnosis raises walking success from 259/640 to 429/640 episodes; online repair reaches 382/640, with no triggers in 64 held-out fault-free episodes. A …
@misc{ziegler2026when,
title = {{When Black-Box Robot Policies Fail: Online Sensor Diagnosis and Repair with a Transparent Surrogate}},
author = {Marc Ziegler and Batu \"Ozen and Sami Mentes and Thomas Waschulzik and Alois Knoll},
year = {2026},
eprint = {2610.36852},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36852},
}
While velocity-commanded RL policies are effective for controlling quadrupeds, they are a limited interface for humanoids that need to perform complex whole-body motions. Furthermore, RL tracking policies for whole-body motions typically have limited information about the desired future motion, preventing them from performing controlled deviations to achieve long-term task success; such information is particularly relevant when a controller must prepare for transitions between behaviors. We address these challenges using concise motion representations that can be generated by a high-level …
Fanqi Yu, Matteo Tiezzi, Cigdem Beyan, Tommaso Apicella et al.
Abstract摘要
Robotic agents operating in real-world environments must adapt continuously from a stream of multimodal demonstrations, often under constraints that preclude storing or revisiting past data. In this replay-free, single-pass lifelong imitation learning setting, sequential updates can progressively distort the representation space, making expert-based approaches particularly sensitive to routing interference, causing expert selection to degrade over time. We propose HIDRA, a structured routing framework that mitigates this issue through a hierarchical dual-attention mechanism, which decouples …
Andrea Vasco Grieco, Evelyn D'Elia, Edoardo Caldarelli, Raffaello Camoriano
Abstract摘要
Deep reinforcement learning (RL) algorithms have been observed to lose the ability to adaptively learn from new data, a phenomenon termed Loss of Plasticity (LoP). While many metrics have been shown to correlate with it, the causal mechanisms and the geometric conditions from which it can arise have not been clearly identified. We analyze LoP in Proximal Policy Optimization with isotropic Gaussian policies, the state-of-the-art formulation for many continuous action spaces in RL. We report multiple metrics which characterize the effect that the policy standard deviation $\sigma$ has on LoP, …
@misc{grieco2026bridging,
title = {{Bridging Exploration and Loss of Plasticity in On-Policy Reinforcement Learning}},
author = {Andrea Vasco Grieco and Evelyn D'Elia and Edoardo Caldarelli and Raffaello Camoriano},
year = {2026},
eprint = {2610.36309},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36309},
}
Jiayi Luo, Yang Zhang, Liliang Chen, Nan Wang et al.
Abstract摘要
Action-conditioned world models offer a scalable alternative to costly real-world interaction for improving robot policies. However, existing world-model-based methods largely treat world models as forward-only surrogates of the physical environment, leaving untapped a capability no physical environment affords: rewinding to past decision points to explore counterfactual futures. We introduce Rollback, a policy improvement paradigm that turns world models from passive simulators into counterfactual engines: it rolls back to critical decision points, branches into alternative actions, and …
@misc{luo2026rollback,
title = {{Rollback: Experience Backtracking in World Models for Policy Improvement}},
author = {Jiayi Luo and Yang Zhang and Liliang Chen and Nan Wang and Chen Gao and Boxiang Qiu and Zhuoxuan Li and Baorui Peng and others},
year = {2026},
eprint = {2610.36068},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.36068},
}
Gijeong Kim, Seungbin Oh, Dongyun Kang, Hae-Won Park
Abstract摘要
This paper proposes a smooth analytic gradient for frictional contact and applies it to policy learning and contact-implicit trajectory optimization for legged locomotion. The exact contact sensitivity is available in closed form but describes changes within the current contact mode, providing no first-order information toward neighboring contact modes. The proposed gradient replaces this sensitivity only in the backward pass, leaving the hard-contact forward solve unchanged and the realized rollout feasible. A central-path relaxation of the de Saxc\'e-corrected second-order-cone …
@misc{kim2026differentiable,
title = {{Differentiable Contact Dynamics for Legged Locomotion: Policy Learning and Contact-Implicit Trajectory Optimization}},
author = {Gijeong Kim and Seungbin Oh and Dongyun Kang and Hae-Won Park},
year = {2026},
eprint = {2610.35852},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35852},
}
Multi-objective reinforcement learning is a fundamental problem for modern AI, with applications spanning post-training, reasoning models, and robotic control. A central challenge is to learn policies that balance competing objectives, such as helpfulness versus honesty in safety and alignment, or speed versus energy in robotics. The optimal policy can vary sharply with objective-weighting combinations, and the extent of this variation depends crucially on the degree of conflict among objectives, making per-preference policy learning sample-inefficient. In this paper, we introduce …
Generalist robot policies and language-model agents provide complementary capabilities, but completing a task also requires adapting their use to the consequences of physical interaction. We study execution as a locus of adaptation: feedback guides the current action and supplies evidence for revising subsequent decisions under fixed model weights. We introduce RoboDoctor, a training-free framework that connects diagnosis, grounded action, and persistent experience across three revision targets. The harness allocates control between policy and agent; intervention determines when to assist …
@misc{yu2026robodoctor,
title = {{RoboDoctor: Training-Free Recursive Self-Improvement for Robot Policies}},
author = {Heyang Yu and Haomin Li and Sihang Li and Jixin Yan and Shiyun Peng and Wanqi Zhong and Tianyu Xin and Yuan Yao and others},
year = {2026},
eprint = {2610.35604},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35604},
}
Yuanzhi He, Victor Romero-Cano, Jiayu Xu, Juan David Hern\'andez et al.
Abstract摘要
Training robotic manipulation policies in simulation is widely adopted due to safety and cost efficiency; however, tasks involving obstacles remain under-performing and often converge slowly under Deep Reinforcement Learning (DRL). To address this challenge, we propose a task decomposition framework for DRL that integrates Transfer Learning (TL) with Reward Redesign (RR) to improve learning efficiency and robustness in obstacle environments. The TL strategy selectively transfers only the first-layer critic parameters across structurally related tasks, since that layer captures general …
@misc{he2026enhancing,
title = {{Enhancing Deep Reinforcement Learning for Robotic Motion Planning via Task Decomposition}},
author = {Yuanzhi He and Victor Romero-Cano and Jiayu Xu and Juan David Hern\'andez and Gualtiero Colombo},
year = {2026},
eprint = {2610.35494},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.35494},
}
Haoyang Jiang, Dana Kulic, Michael Burke, Pamela Carreno-Medrano
Abstract摘要
Human behaviour models are increasingly used to provide simulated human interactions for training robots via reinforcement learning (RL). Existing approaches primarily evaluate these models according to how closely their generated behaviours resemble real human data, while their usefulness as training partners is less commonly assessed. This creates a potential objective mismatch: a model that performs well according to behavioural evaluation metrics may not necessarily provide useful training experiences for downstream robot policy learning. Across two human-robot interaction (HRI) domains, …
@misc{jiang2026behavioural,
title = {{Behavioural Quality Is Not Learning Utility: Objective Mismatch in Human Behaviour Models for Robot Learning}},
author = {Haoyang Jiang and Dana Kulic and Michael Burke and Pamela Carreno-Medrano},
year = {2026},
eprint = {2610.34839},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34839},
}
Achieving safety in autonomous driving through Multi-Agent Reinforcement Learning (MARL) is a critical yet challenging task due to non-stationarity, partial observability, and the need for effective coordination among agents. Although earlier cooperative MARL methods have aimed to improve coordination by sharing encoded state observations, we find these strategies are insufficient in safety-critical scenarios. To address this gap, we present IntNet, a novel MARL framework that enhances safety by incorporating the transmission of vehicle intent and adaptive communication scheduling into a …
Endowing robot policies with anticipatory capabilities has proven effective in improving control performance. However, existing methods predominantly rely on explicit supervision derived from future observations, in either pixel or latent space, at the cost of additional architectural and computational complexity. Departing from explicit future supervision, we ask whether such foresight can be elicited directly from the action signal. To this end, we propose \textbf{AdaPolicy}, an \textbf{A}ction-\textbf{d}riven \textbf{a}nticipatory policy framework that shapes future-aware representations …
@misc{yang2026adapolicy,
title = {{AdaPolicy: Action-Driven Anticipatory Robot Policies via Next-Chunk Prediction}},
author = {Fan Yang and Yuting Su and Chen Zhao and Fugui Fan and Peiguang Jing},
year = {2026},
eprint = {2610.34280},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.34280},
}
Xuebin Li, Jinpeng Xiao, Zong Chen, Pierluigi Arpenti et al.
Abstract摘要
Cooperative transport with heterogeneous legged robots requires coordinating different motion capabilities while supporting an unfixed shared payload. We present HA-DiPO (Heterogeneous-Agent Diffusion Policy Optimization), a hierarchical multi-agent reinforcement learning framework under centralized training with decentralized execution (CTDE). The framework decouples team-level coordination from embodiment-specific motion control: at the high level, a centralized critic uses team-value estimates to jointly train independent coordination policies, aligning robot-specific action responses with …
@misc{li2026ha,
title = {{HA-DiPO: Diffusion Policy Optimization for Heterogeneous Cooperative Transport}},
author = {Xuebin Li and Jinpeng Xiao and Zong Chen and Pierluigi Arpenti and Bruno Siciliano and Jiebao Sun and Yiqun Li},
year = {2026},
eprint = {2610.33584},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.33584},
}
Zheng Zhang, Xiaoyang Tang, Shuaiqi Cheng, Jiakai Huang et al.
Abstract摘要
Robot policy evaluation informs deployment decisions and policy improvement, but physical testing and simulator preparation require substantial effort. World models offer a complementary route through closed-loop interaction in generated environments. Recent benchmarks have demonstrated their potential as policy evaluators through outcome-based evaluation (i.e., assessing task success or failure). However, such evaluation does not establish whether world models faithfully simulate how policies fail. Misrepresenting failures can lead to incorrect diagnoses of policy weaknesses and make …
@misc{zhang2026worldfailbench,
title = {{WorldFailBench: Do World Models Understand Policy Failures?}},
author = {Zheng Zhang and Xiaoyang Tang and Shuaiqi Cheng and Jiakai Huang and Zhongbo Wu and Ziyi Zhang and Yuan Si and Zeyu Liu and others},
year = {2026},
eprint = {2610.33565},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.33565},
}
Zongyou Yu, Ao Zhou, Xiongbo Shi, Jiayi Guan et al.
Abstract摘要
Dexterous manipulation requires robots to leverage existing datasets for rapid skill acquisition while maintaining safety during real-world deployment to prevent damage to the robot or its surroundings. Offline-to-online safe reinforcement learning (O2O safe RL) provides a promising solution for this problem by leveraging prior experience for online improvement under safety constraints, thereby mitigating the sample inefficiency and exploration risk of learning safe policies from scratch. However, existing O2O safe RL approaches do not explicitly account for the safety budget already consumed …
@misc{yu2026rbga,
title = {{RBGA: Residual-Budget Guide Arbitration for Offline-to-Online Safe Reinforcement Learning in Dexterous Manipulation}},
author = {Zongyou Yu and Ao Zhou and Xiongbo Shi and Jiayi Guan and Shihang Du and Xudong Zhang and Shangding Gu and Junqiao Zhao and others},
year = {2026},
eprint = {2610.33494},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2610.33494},
}
Dexterous manipulation requires high-dimensional control policies to accomplish complex contact-rich tasks while simultaneously satisfying multiple safety requirements, such as joint limits, unsafe finger configurations, arm collisions, and unstable object interactions. Constrained reinforcement learning (RL) provides a principled framework for incorporating these requirements into dexterous manipulation policy learning. However, existing constrained RL methods for dexterous manipulation primarily focus on cumulative cost constraints and struggle to simultaneously handle multiple …
@misc{zhou2026mdpo,
title = {{MDPO: Multi-constraint Dexterous Policy Optimization with Reinforcement Learning}},
author = {Ao Zhou and Zongyou Yu and Ke Wu and Tao Xu and Junqiao Zhao and Wenchao Ding and Alois Knoll and Guang Chen},
year = {2026},
eprint = {2610.33480},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.33480},
}
Robust reinforcement learning (RL) is critical for reliable real-world deployment, where policies may fail under diverse shifts and disruptions. Existing robust RL algorithms offer a rich set of mechanisms and principles, yet two major barriers limit their broader potential: limited algorithm reusability and an unclear empirical capability boundary. We introduce RobustRLlib, an algorithm-centric library and benchmark that addresses both challenges. It integrates 17 representative robust online, offline, and safe RL algorithms under a unified interface for substantially improved reusability, …
@misc{chen2026robustrllib,
title = {{RobustRLlib: A Unified Library and Benchmark for Robust Reinforcement Learning Algorithms}},
author = {Xi Chen and Fengqi Hu and Zeyu Shen and Jiaying Xu and Shangding Gu and Pan Xu and Anqi Liu and Laixi Shi},
year = {2026},
eprint = {2610.32971},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.32971},
}
Danilo Vucetic, Mohamad H. Danesh, Florence Clerc, Hsiu-Chin Lin et al.
Abstract摘要
Regularised reinforcement learning (RL) has provable robustness guarantees that should ease the transfer of policies from simulation to robots, known as sim-to-real. Yet, weakly regularised algorithms like Proximal Policy Optimisation (PPO) remain the de facto choice in sim-to-real pipelines, while the practical benefits of regularisation for transfer remain poorly characterised. In this work, we study the sim-to-real performance of commonly used RL algorithms, with the aim of characterising the effects of regularisation on robotic performance. To that end, we derive Soft MellowMax …
@misc{vucetic2026regularised,
title = {{Regularised RL improves sim-to-real transfer}},
author = {Danilo Vucetic and Mohamad H. Danesh and Florence Clerc and Hsiu-Chin Lin and Jonathan Tremblay},
year = {2026},
eprint = {2610.32855},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.32855},
}
These lecture notes provide an introduction to optimization, optimal control, and reinforcement learning for robotics, emphasizing their mathematical connections and practical relevance. They present established material compiled and edited by Yonghyeon Lee, rather than new research results. The notes were developed through a human and AI collaborative writing process, with AI assistance in drafting, organizing, and refining the exposition. This first version is a preliminary collection of notes, not yet a fully developed textbook, and currently contains no figures or graphs. The notes may be …
Shouhao Chang, Xuan Liu, Hongye Zhu, Xinning Chen et al.
Abstract摘要
Designing effective reward functions is a critical challenge in Reinforcement Learning (RL), which traditionally requires costly manual trial-and-error. While Large Language Model (LLM)-based methods have shown promise in reward design, undifferentiated sampling and limited use of historical feedback often lead to insufficient exploration of the reward space and optimization instability. To address these challenges, we propose $R^2E$, a Role-driven Reward Evolutionary framework for automated reward function design that structures the process through explicit role specialization. $R^2E$ …
@misc{chang2026r,
title = {{$R^2E:$ A Role-driven Reward Evolutionary Framework for Automated Reward Function Design}},
author = {Shouhao Chang and Xuan Liu and Hongye Zhu and Xinning Chen and Shigeng Zhang},
year = {2026},
eprint = {2610.32439},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.32439},
}
Zongyao Zhao, Cheng-Jie Zhao, Ruihang Jiang, Bing Zheng et al.
Abstract摘要
A failed vision-language-action (VLA) rollout does not imply that every action contributed to failure. Yet in group-relative policy optimization (GRPO), a single trajectory-level advantage is assigned to all actions in a rollout. Potentially useful recovery attempts can thus be penalized alongside the errors they aim to correct. Ideal stopping conditions for failure feedback are characterized using success-probability and policy-score martingales. At these stops, the removed gradient contribution has zero conditional expectation given the retained information. Truncation therefore preserves …
@misc{zhao2026when,
title = {{When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning}},
author = {Zongyao Zhao and Cheng-Jie Zhao and Ruihang Jiang and Bing Zheng and Zhaolin Wang and Yuanwei Liu},
year = {2026},
eprint = {2610.32431},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2610.32431},
}
Recent robot manipulation policies commonly predict continuous action chunks directly from observations, without explicitly modeling reusable structure in the action space. However, manipulation demonstrations often contain recurring short-horizon motion patterns that are shared across tasks. We introduce \textsc{RoboCode}, which learns these patterns through a shared soft-gated primitive action interface. Rather than assigning each action chunk to a single discrete code, \textsc{RoboCode} represents it as a soft composition of multiple learned primitives, providing a flexible intermediate …
Takuma Tsukakoshi, Elle Miller, Tamon Miyake, Namiko Saito et al.
Abstract摘要
Robot-assisted dressing is particularly challenging due to narrow openings and articulated body parts. Even small errors in the relative pose between the body part and garment opening can cause snagging, missed insertion, or incomplete dressing. To address these challenges, we formulate dressing as a geometry-aware insertion problem that represents task-relevant spatial relationships between the body part and object opening, rather than relying on the full deformable-object state. Based on this formulation, we propose a Reinforcement Learning (RL) framework combining a staged curriculum, …
@misc{tsukakoshi2026geometry,
title = {{Geometry-Aware Reinforcement Learning for Close-Fitting Wearable Insertion}},
author = {Takuma Tsukakoshi and Elle Miller and Tamon Miyake and Namiko Saito and Yushi Wang and Shigeki Sugano and Tetsuya Ogata and Sethu Vijayakumar},
year = {2026},
eprint = {2610.32143},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.32143},
}
Minho Park, Kinam Kim, Donghu Kim, Byungkun Lee et al.
Abstract摘要
Behavior cloning for robotic manipulation is highly sensitive to the coverage of its training data, yet collecting diverse successful demonstrations remains costly. We study reinforcement learning (RL) not as a deployable controller, but as a data generator for downstream visuomotor imitation learning. Our key premise is that multiple successful solution modes cover a broader range of successful trajectories than repeatedly collecting a single narrow solution. We introduce DiscoDemo, which trains a privileged latent-conditioned RL generator with a parallel reverse curriculum and a …
@misc{park2026discodemo,
title = {{DiscoDemo: Discovering Efficient and Diverse Robot Demonstrations for Imitation Learning}},
author = {Minho Park and Kinam Kim and Donghu Kim and Byungkun Lee and Dongyoon Hwang and Yongjae Shin and Junha Hyung and Hojoon Lee and others},
year = {2026},
eprint = {2610.32093},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.32093},
}
Chubin Zhang, Changyuan Wang, Yirui Wang, Ao Li et al.
Abstract摘要
Scaling policy learning requires broad experience and reliable supervision. Robot demonstrations provide high-quality trajectory supervision but are costly to collect. Egocentric videos broaden task and environment coverage, yet motion reconstruction and retargeting introduce noisy action labels. To address this challenge, we introduce EgoForesight, a vision-language-action framework that learns jointly from both sources by reducing the influence of unreliable action labels and exploiting demonstrated outcomes as complementary supervision. A joint vision-action flow couples future visual …
@misc{zhang2026egoforesight,
title = {{EgoForesight: Effectively Scaling Policy Learning with Noisy Egocentric Actions}},
author = {Chubin Zhang and Changyuan Wang and Yirui Wang and Ao Li and Shilin Ma and Yue Su and Cai Zhou and Xiuwei Xu and others},
year = {2026},
eprint = {2610.31845},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31845},
}
Real-world reinforcement learning has enabled contact-rich manipulation tasks to be trained directly on physical robots, but unsafe interactions during online learning remain a key barrier to practical deployment. We propose Safe Evidential Real-World Reinforcement Learning (SEVI), a framework for reliability-aware safety intervention during real-world policy learning. SEVI introduces an Evidential Safety Critic that estimates both future constraint-violation risk and epistemic uncertainty in a single forward pass. During online learning, SEVI evaluates each task-policy action before …
Ruiqi Wang, Dezhong Zhao, Xuesu Xiao, Erdem Biyik et al.
Abstract摘要
Preference-based reinforcement learning (PbRL) has emerged as a promising paradigm for learning robot behaviors aligned with human intent. However, its scalability is fundamentally limited by the need to collect expensive human preference labels for every new task. In this work, we present PrefO2O, an offline-to-online latent preference transfer framework that enables zero-shot PbRL on unseen target tasks without requiring any human feedback. The core idea is to learn a task-invariant latent preference space from a small amount of offline preference data on source tasks, where transferable …
@misc{wang2026prefo,
title = {{PrefO2O: Zero-Shot Preference-Based Reinforcement Learning via Offline-to-Online Latent Preference Transfer}},
author = {Ruiqi Wang and Dezhong Zhao and Xuesu Xiao and Erdem Biyik and Byung-Cheol Min and Guohua Chen},
year = {2026},
eprint = {2610.31812},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31812},
}
Fanxing Li, Yuxiang Huang, Shengyang Wang, Fangyu Sun et al.
Abstract摘要
Tasks such as quadrotor dynamic-gate racing require agents to anticipate external motion while maintaining precise high-frequency control. World-model-based reinforcement learning provides a natural solution by learning latent dynamics for future imagination, yet it often entangles environmental evolution and the robot's own dynamics in a single learned rollout, where model bias can affect policy optimization efficiency. We propose DiffDreamer, a differentiable reinforcement learning framework that tightly couples a learned world model with differentiable simulation for actor optimization. In …
@misc{li2026diffdreamer,
title = {{DiffDreamer: Coupling World Models with Differentiable Simulation for Agile Quadrotor Control}},
author = {Fanxing Li and Yuxiang Huang and Shengyang Wang and Fangyu Sun and Shuyu Wu and Tianbao Zhang and Dexin Zuo and Yan Yufei and others},
year = {2026},
eprint = {2610.31617},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31617},
}