NVIDIA · embodied AI & robot learning英伟达 · 具身智能与机器人学习
36 curated papers, code, datasets and hardware from NVIDIA (26 with code), with one-line takeaways.36 个来自英伟达的精选论文、代码、数据集与硬件（26 个附代码），均配一句话要点。
Labs are matched from each entry's affiliation field by simple name rules (data/labs.yaml); joint work appears under every lab named. Only curated entries are listed, because daily arXiv papers carry no affiliations.机构由每个条目的机构字段按名称规则匹配（data/labs.yaml），合作工作会出现在所有署名机构下。仅列出精选条目，因为每日 arXiv 论文不含机构信息。
GPU-parallel motion generation for arms — many trajectory optimizations run at once, giving collision-free, smooth plans in milliseconds.面向机械臂的 GPU 并行运动生成：同时运行大量轨迹优化，在毫秒级给出无碰撞且平滑的轨迹。
@misc{sundaralingam2023curobo,
title = {{cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation}},
author = {Balakumar Sundaralingam and Siva Kumar Sastry Hari and Adam Fishman and Caelan Garrett and Karl Van Wyk and Valts Blukis and Alexander Millane and Helen Oleynikova and Ankur Handa and Fabio Ramos and Nathan Ratliff and Dieter Fox},
year = {2023},
eprint = {2310.17274},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2310.17274},
}
Thousands of parallel GPU environments train a quadruped walking policy in minutes; legged_gym became a standard starter.数千个 GPU 并行环境几分钟就能训练出四足行走策略；legged_gym 成为标准入门框架。
@misc{rudin2021learning,
title = {{Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning}},
author = {Nikita Rudin and David Hoeller and Philipp Reist and Marco Hutter},
year = {2021},
eprint = {2109.11978},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2109.11978},
}
Continual off-policy RL for VLAs — small actor/critic networks carry over between tasks, distilled on a few kept episodes to avoid forgetting; reports 31–68.5% less real-robot time on new assembly tasks.面向 VLA 的持续离策略强化学习：轻量 actor/critic 在任务间延续，用少量保留的旧任务轨迹做蒸馏以防遗忘。论文报告新的接触式装配任务所需真机训练时间减少 31–68.5%，旧任务几乎不遗忘。
@misc{kim2026accrue,
title = {{ACCRUE: Continual Off-Policy Reinforcement Learning for Robot Foundation Models}},
author = {Changyeon Kim and Kimin Lee and Yashraj Narang and Yijie Guo},
year = {2026},
eprint = {2610.18596},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.18596},
}
The open data engine behind GR00T training: models that write timestamped action captions, dense process rewards and episode outcome rewards for human and robot video; reports SOTA on WGO-Bench, RoboReward and Robometer.GR00T 训练所用的开放数据引擎：一组模型为人类与机器人视频生成带时间戳的动作描述、稠密过程奖励和整段结果奖励；报告在 WGO-Bench、RoboReward 与 Robometer 上达到最优。
@misc{leng2026flora,
title = {{FLORA: An Open Model Suite for Scaling Robotics Data Engines}},
author = {Sicong Leng and Yunze Man and Zhiqi Li and Jing Wang and Yuhe Zhang and Ryan Punamiya and Siddharth Patki and Kaiyuan Zheng and Zi-ang Cao and Zhiding Yu and Sifei Liu and Danfei Xu and Ben Levin and Jan Kautz and Yuke Zhu and Johan Bjorck and Linxi "Jim" Fan},
year = {2026},
eprint = {2610.31779},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.31779},
}
VLA that reasons in discrete motion tokens (a chain of dynamics) shared across embodiments; predicted futures are scored against what actually happened, so action-free human videos give RL signal. Sim and real tests.VLA 用跨本体共享的离散运动 token 链（Chain-of-Dynamics）表达动作思考；预测每条链导致的未来并与真实结果比对来强化，使没有动作标注的人类视频也能提供监督。在仿真与真机上验证人到机器人的迁移。
@misc{kim2026thinkworld,
title = {{ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning}},
author = {Chaehyun Kim and Chi-Pin Huang and Jisoo Kim and Min-Hung Chen and Yu-Chiang Frank Wang and Seungryong Kim and Fu-En Yang},
year = {2026},
eprint = {2610.37666},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37666},
}
Learns a residual "delta action" model from real rollouts to fix sim physics, enabling agile athletic humanoid motions.从真实轨迹学习残差"增量动作"模型来修正仿真物理，实现敏捷的人形运动技能。
@misc{he2025asap,
title = {{ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills}},
author = {Tairan He and Jiawei Gao and Wenli Xiao and Yuanhang Zhang and Zi Wang and Jiashun Wang and Zhengyi Luo and Guanqi He and Nikhil Sobanbab and Chaoyi Pan and Zeji Yi and Guannan Qu and Kris Kitani and Jessica Hodgins and Linxi "Jim" Fan and Yuke Zhu and Changliu Liu and Guanya Shi},
year = {2025},
eprint = {2502.01143},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2502.01143},
}
@misc{nvidia2025cosmos,
title = {{Cosmos World Foundation Model Platform for Physical AI}},
author = {NVIDIA and : and Niket Agarwal and Arslan Ali and Maciej Bala and Yogesh Balaji and Erik Barker and Tiffany Cai and Prithvijit Chattopadhyay and Yongxin Chen and Yin Cui and Yifan Ding and Daniel Dworakowski and Jiaojiao Fan and Michele Fenzi and Francesco Ferroni and Sanja Fidler and Dieter Fox and Songwei Ge and Yunhao Ge and Jinwei Gu and Siddharth Gururani and Ethan He and Jiahui Huang and Jacob Huffman and Pooya Jannaty and Jingyi Jin and Seung Wook Kim and Gergely Klár and Grace Lam and Shiyi Lan and Laura Leal-Taixe and Anqi Li and Zhaoshuo Li and Chen-Hsuan Lin and Tsung-Yi Lin and Huan Ling and Ming-Yu Liu and Xian Liu and Alice Luo and Qianli Ma and Hanzi Mao and Kaichun Mo and Arsalan Mousavian and Seungjun Nah and Sriharsha Niverty and David Page and Despoina Paschalidou and Zeeshan Patel and Lindsey Pavao and others},
year = {2025},
eprint = {2501.03575},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2501.03575},
}
DexUMI — an exoskeleton matched to the robot hand's fingertip workspace, plus inpainting that swaps the human hand for the robot's; reports 86% average success (Inspire, XHand) and ~3.2× faster collection than teleop.DexUMI：按目标机器人手的指尖工作空间设计的可穿戴手部外骨骼，再用视频修补把画面中的人手换成机器人手；论文报告在 Inspire 与 XHand 上平均成功率 86%，采集效率约为遥操作的 3.2 倍。
@misc{xu2025dexumi,
title = {{DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation}},
author = {Mengda Xu and Han Zhang and Yifan Hou and Zhenjia Xu and Linxi Fan and Manuela Veloso and Shuran Song},
year = {2025},
eprint = {2505.21864},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2505.21864},
}
Fine-tunes video world models on a robot to dream new demos, labels them with latent actions or an inverse dynamics model, and trains policies on them; a humanoid learns 20+ new behaviors from one-scene teleop data.把视频世界模型微调到目标机器人上“做梦”生成新示范，再用隐动作或逆动力学模型补上动作标签用于训练策略；人形机器人仅凭单一场景的遥操作数据学会 20 多种新行为。
@misc{jang2025dreamgen,
title = {{DreamGen: Unlocking Generalization in Robot Learning through Video World Models}},
author = {Joel Jang and Seonghyeon Ye and Zongyu Lin and Jiannan Xiang and Johan Bjorck and Yu Fang and Fengyuan Hu and Spencer Huang and Kaushil Kundalia and Yen-Chen Lin and Loic Magne and Ajay Mandlekar and Avnish Narayan and You Liang Tan and Guanzhi Wang and Jing Wang and Qi Wang and Yinzhen Xu and Xiaohui Zeng and Kaiyuan Zheng and Ruijie Zheng and Ming-Yu Liu and Luke Zettlemoyer and Dieter Fox and Jan Kautz and Scott Reed and Yuke Zhu and Linxi Fan},
year = {2025},
eprint = {2505.12705},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2505.12705},
}
Stereo depth model trained on large synthetic data that reports strong zero-shot accuracy on real scenes — a drop-in depth source for stereo-camera robots.在大规模合成数据上训练的双目深度模型，报告称在真实场景上零样本精度很强；可直接作为双目相机机器人的深度来源。
@misc{wen2025foundationstereo,
title = {{FoundationStereo: Zero-Shot Stereo Matching}},
author = {Bowen Wen and Matthew Trepte and Joseph Aribido and Jan Kautz and Orazio Gallo and Stan Birchfield},
year = {2025},
eprint = {2501.09898},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2501.09898},
}
@misc{nvidia2025gr,
title = {{GR00T N1: An Open Foundation Model for Generalist Humanoid Robots}},
author = {NVIDIA and : and Johan Bjorck and Fernando Castañeda and Nikita Cherniadev and Xingye Da and Runyu Ding and Linxi "Jim" Fan and Yu Fang and Dieter Fox and Fengyuan Hu and Spencer Huang and Joel Jang and Zhenyu Jiang and Jan Kautz and Kaushil Kundalia and Lawrence Lao and Zhiqi Li and Zongyu Lin and Kevin Lin and Guilin Liu and Edith Llontop and Loic Magne and Ajay Mandlekar and Avnish Narayan and Soroush Nasiriany and Scott Reed and You Liang Tan and Guanzhi Wang and Zu Wang and Jing Wang and Qi Wang and Jiannan Xiang and Yuqi Xie and Yinzhen Xu and Zhenjia Xu and Seonghyeon Ye and Zhiding Yu and Ao Zhang and Hao Zhang and Yizhou Zhao and Ruijie Zheng and Yuke Zhu},
year = {2025},
eprint = {2503.14734},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2503.14734},
}
A hierarchical VLA in which a fine-tuned VLM draws a coarse 2D path for the end effector on the image and a low-level 3D policy follows it, so the VLM can be trained on cheap off-domain data such as sim or videos.分层 VLA：微调后的 VLM 在图像上画出末端执行器的粗略二维路径，再由底层三维策略跟随执行；因此 VLM 可以用仿真、视频等低成本的非本域数据训练。
@misc{li2025hamster,
title = {{HAMSTER: Hierarchical Action Models for Open-World Robot Manipulation}},
author = {Yi Li and Yuquan Deng and Jesse Zhang and Joel Jang and Marius Memmel and Raymond Yu and Caelan Reed Garrett and Fabio Ramos and Dieter Fox and Anqi Li and Abhishek Gupta and Ankit Goyal},
year = {2025},
eprint = {2502.05485},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2502.05485},
}
The reference paper for Isaac Lab, Orbit's successor — GPU-parallel physics, rendering and sensor simulation with ready-made robots and tasks for RL and imitation.Isaac Lab（Orbit 的后继）的正式论文：GPU 并行的物理、渲染与传感器仿真，内置机器人与任务，面向强化学习和模仿学习。
@misc{nvidia2025isaac,
title = {{Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning}},
author = {NVIDIA and : and Mayank Mittal and Pascal Roth and James Tigue and Antoine Richard and Octi Zhang and Peter Du and Antonio Serrano-Muñoz and Xinjie Yao and René Zurbrügg and Nikita Rudin and Lukasz Wawrzyniak and Milad Rakhsha and Alain Denzler and Eric Heiden and Ales Borovicka and Ossama Ahmed and Iretiayo Akinola and Abrar Anwar and Mark T. Carlson and Ji Yuan Feng and Animesh Garg and Renato Gasoto and Lionel Gulich and Yijie Guo and M. Gussert and Alex Hansen and Mihir Kulkarni and Chenran Li and Wei Liu and Viktor Makoviychuk and Grzegorz Malczyk and Hammad Mazhar and Masoud Moghani and Adithyavairavan Murali and Michael Noseworthy and Alexander Poddubny and Nathan Ratliff and Welf Rehberg and Clemens Schwarke and Ritvik Singh and James Latham Smith and Bingjie Tang and Ruchik Thaker and Matthew Trepte and Karl Van Wyk and Fangzhou Yu and Alex Millane and Vikram Ramasamy and others},
year = {2025},
eprint = {2511.04831},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2511.04831},
}
Open, GPU-accelerated physics engine built on NVIDIA Warp and OpenUSD with multiple solvers for contact-rich robot learning; co-developed by Disney Research, Google DeepMind and NVIDIA, hosted by the Linux Foundation.基于 NVIDIA Warp 和 OpenUSD 的开源 GPU 物理引擎，支持多种求解器，面向接触丰富的机器人学习；由 Disney Research、Google DeepMind 和 NVIDIA 联合开发，由 Linux 基金会托管。
Extends MimicGen to humanoids with two dexterous hands, turning 60 human demos into 21K simulated ones across bimanual tasks, plus a real-to-sim-to-real pipeline tested on real can sorting.把 MimicGen 扩展到双灵巧手人形机器人，从 60 条人类演示生成 2.1 万条双臂仿真演示，并给出在真实分拣罐子任务上验证的 real-to-sim-to-real 流程。
@misc{jiang2024dexmimicgen,
title = {{DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning}},
author = {Zhenyu Jiang and Yuqi Xie and Kevin Lin and Zhenjia Xu and Weikang Wan and Ajay Mandlekar and Linxi Fan and Yuke Zhu},
year = {2024},
eprint = {2410.24185},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.24185},
}
Distills one policy that supports several humanoid control modes (root velocity, joint angles, keypoint tracking) instead of training a separate controller per mode.蒸馏出单一策略，支持多种人形控制模式（根部速度、关节角、关键点跟踪），不必为每种模式各训练一个控制器。
@misc{he2024hover,
title = {{HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots}},
author = {Tairan He and Wenli Xiao and Toru Lin and Zhengyi Luo and Zhenjia Xu and Zhenyu Jiang and Jan Kautz and Changliu Liu and Guanya Shi and Xiaolong Wang and Linxi Fan and Yuke Zhu},
year = {2024},
eprint = {2410.21229},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.21229},
}
LAPA pretrains a VLA without action labels: it learns discrete latent actions between video frames, predicts them from image + text, then maps them to robot actions; reports beating OpenVLA on real tasks.LAPA 无需动作标注即可预训练 VLA：先从视频相邻帧间学习离散隐动作，再训练模型由图像和指令预测隐动作，最后用少量机器人数据映射到真实动作；报告在真机任务上优于 OpenVLA。
@misc{ye2024latent,
title = {{Latent Action Pretraining from Videos}},
author = {Seonghyeon Ye and Joel Jang and Byeongguk Jeon and Sejune Joo and Jianwei Yang and Baolin Peng and Ajay Mandlekar and Reuben Tan and Yu-Wei Chao and Bill Yuchen Lin and Lars Liden and Kimin Lee and Jianfeng Gao and Luke Zettlemoyer and Dieter Fox and Minjoon Seo},
year = {2024},
eprint = {2410.11758},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.11758},
}
A VLA outputs mid-level navigation commands in language that an RL locomotion policy executes, so legged robots follow natural-language routes; adds Isaac Lab VLN benchmarks.VLA 先用语言输出中层导航指令，再由强化学习运动策略执行，让腿式机器人按自然语言指令导航；并提供 Isaac Lab 上的 VLN 基准。
@misc{cheng2024navila,
title = {{NaVILA: Legged Robot Vision-Language-Action Model for Navigation}},
author = {An-Chieh Cheng and Yandong Ji and Zhaojing Yang and Zaitian Gongye and Xueyan Zou and Jan Kautz and Erdem Bıyık and Hongxu Yin and Sifei Liu and Xiaolong Wang},
year = {2024},
eprint = {2412.04453},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2412.04453},
}
120 kitchen scenes, 2,500+ objects and 100 tasks with generated demonstrations for training generalist policies.120 个厨房场景、2500 多个物体与 100 个任务，并附生成演示，用于训练通用策略。
@misc{nasiriany2024robocasa,
title = {{RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots}},
author = {Soroush Nasiriany and Abhiram Maddukuri and Lance Zhang and Adeet Parikh and Aaron Lo and Abhishek Joshi and Ajay Mandlekar and Yuke Zhu},
year = {2024},
eprint = {2406.02523},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2406.02523},
}
@misc{pumacay2024colosseum,
title = {{THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation}},
author = {Wilbert Pumacay and Ishika Singh and Jiafei Duan and Ranjay Krishna and Jesse Thomason and Dieter Fox},
year = {2024},
eprint = {2402.08191},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2402.08191},
}
GPU-based simulation of GelSight-style tactile images and contact forces in Isaac Gym, plus learning tools for sim-to-real transfer of visuotactile policies.在 Isaac Gym 中用 GPU 模拟 GelSight 式触觉图像与接触力，并提供用于视触觉策略仿真到真实迁移的学习工具。
@misc{akinola2024tacsl,
title = {{TacSL: A Library for Visuotactile Sensor Simulation and Learning}},
author = {Iretiayo Akinola and Jie Xu and Jan Carius and Dieter Fox and Yashraj Narang},
year = {2024},
eprint = {2408.06506},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2408.06506},
}
A learned navigation policy sequences pre-trained skills (walk, jump, climb, crouch) using a learned obstacle reconstruction; reports up to 2 m/s on hardware (Science Robotics).分层的全学习系统：导航策略结合学习到的障碍物重建来调度预训练技能（行走、跳跃、攀爬、下蹲）；报告实机速度可达 2 米/秒。发表于 Science Robotics。
Camera-only teleoperation for arm + dexterous hand pairs - tracks the operator's hand and retargets it to many different robot hands, in sim or on real hardware.仅用相机即可遥操作"机械臂 + 灵巧手"：跟踪操作者手部并重定向到多种不同的机器人手，可用于仿真或真机。
@misc{qin2023anyteleop,
title = {{AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System}},
author = {Yuzhe Qin and Wei Yang and Binghao Huang and Karl Van Wyk and Hao Su and Xiaolong Wang and Yu-Wei Chao and Dieter Fox},
year = {2023},
eprint = {2307.04577},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2307.04577},
}
Tracks the 6-DoF pose of a never-seen object in RGB-D video while building its neural 3D model online, with no CAD model needed; useful for manipulating novel objects.在 RGB-D 视频中跟踪从未见过物体的六自由度位姿，同时在线构建其神经三维模型，无需 CAD 模型；适用于操作新物体。
@misc{wen2023bundlesdf,
title = {{BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects}},
author = {Bowen Wen and Jonathan Tremblay and Valts Blukis and Stephen Tyree and Thomas Muller and Alex Evans and Dieter Fox and Jan Kautz and Stan Birchfield},
year = {2023},
eprint = {2303.14158},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2303.14158},
}
6D pose estimation and tracking of unseen objects from a CAD model or a few reference images, with no per-object training.只需 CAD 模型或少量参考图像即可对未见过的物体做 6D 位姿估计与跟踪，无需逐物体训练。
@misc{wen2023foundationpose,
title = {{FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects}},
author = {Bowen Wen and Wei Yang and Jan Kautz and Stan Birchfield},
year = {2023},
eprint = {2312.08344},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2312.08344},
}
Turns a handful of human demos into thousands by splitting them into object-centric segments and replaying those segments in new scenes; widely reused to scale imitation data.把少量人类演示切分为以物体为中心的片段，并在新场景中变换重放，从而扩增为数千条演示；被广泛用于扩充模仿学习数据。
@misc{mandlekar2023mimicgen,
title = {{MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations}},
author = {Ajay Mandlekar and Soroush Nasiriany and Bowen Wen and Iretiayo Akinola and Yashraj Narang and Linxi Fan and Yuke Zhu and Dieter Fox},
year = {2023},
eprint = {2310.17596},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2310.17596},
}
@misc{mittal2023orbit,
title = {{Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments}},
author = {Mayank Mittal and Calvin Yu and Qinxi Yu and Jingzhou Liu and Nikita Rudin and David Hoeller and Jia Lin Yuan and Ritvik Singh and Yunrong Guo and Hammad Mazhar and Ajay Mandlekar and Buck Babich and Gavriel State and Marco Hutter and Animesh Garg},
year = {2023},
eprint = {2301.04195},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2301.04195},
}
RVT — re-renders the point cloud into a few virtual views and uses a multi-view Transformer to predict keyframe poses; reported to train and run much faster than voxel-based PerAct on RLBench. Repo also hosts RVT-2.RVT：把点云重新渲染为若干虚拟视角，再用多视角 Transformer 预测关键帧位姿；论文报告在 RLBench 上训练与推理都比基于体素的 PerAct 快得多。代码库同时包含 RVT-2。
@misc{goyal2023rvt,
title = {{RVT: Robotic View Transformer for 3D Object Manipulation}},
author = {Ankit Goyal and Jie Xu and Yijie Guo and Valts Blukis and Yu-Wei Chao and Dieter Fox},
year = {2023},
eprint = {2306.14896},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2306.14896},
}
A neural policy trained on millions of planner-generated trajectories that moves an arm to goals straight from point clouds.在数百万条规划器生成的轨迹上训练的神经策略，可直接根据点云把机械臂移动到目标位置。
PerAct — a Perceiver Transformer over voxelized RGB-D observations that predicts the next keyframe gripper pose as voxel classification; one language-conditioned policy for many RLBench and real tasks.PerAct：在体素化的 RGB-D 观测上运行 Perceiver Transformer，把下一关键帧夹爪位姿当作体素分类来预测；一个语言条件策略即可覆盖众多 RLBench 与真机任务。
Casts many tabletop tasks as interleaved text-and-image prompts to one Transformer policy, with the VIMA-Bench suite for systematic generalization tests.把多种桌面任务统一成图文交错的提示，交给一个 Transformer 策略执行，并配套 VIMA-Bench 用于系统性泛化评测。
@misc{jiang2022vima,
title = {{VIMA: General Robot Manipulation with Multimodal Prompts}},
author = {Yunfan Jiang and Agrim Gupta and Zichen Zhang and Guanzhi Wang and Yongqiang Dou and Yanjun Chen and Li Fei-Fei and Anima Anandkumar and Yuke Zhu and Linxi Fan},
year = {2022},
eprint = {2210.03094},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2210.03094},
}
CLIPort — combines CLIP's semantic "what" stream with Transporter's spatial "where" stream for language-conditioned tabletop manipulation; an early bridge from web vision-language models to robot actions.CLIPort：把 CLIP 提供的语义"是什么"通路与 Transporter 的空间"在哪里"通路结合，实现语言条件的桌面操作；是较早把网络视觉语言模型接入机器人动作的工作。
@misc{shridhar2021cliport,
title = {{CLIPort: What and Where Pathways for Robotic Manipulation}},
author = {Mohit Shridhar and Lucas Manuelli and Dieter Fox},
year = {2021},
eprint = {2109.12098},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2109.12098},
}
Contact-GraspNet — predicts 6-DoF parallel-jaw grasps for a whole cluttered scene from one depth view by treating observed points as grasp contacts, which shrinks the problem to 4 DoF per point.Contact-GraspNet：从单帧深度图为整个杂乱场景预测 6 自由度平行夹爪抓取；把观测到的点当作抓取接触点，使每个点只需学习 4 个自由度。
@misc{chao2021dexycb,
title = {{DexYCB: A Benchmark for Capturing Hand Grasping of Objects}},
author = {Yu-Wei Chao and Wei Yang and Yu Xiang and Pavlo Molchanov and Ankur Handa and Jonathan Tremblay and Yashraj S. Narang and Karl Van Wyk and Umar Iqbal and Stan Birchfield and Jan Kautz and Dieter Fox},
year = {2021},
eprint = {2104.04631},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2104.04631},
}
@misc{makoviychuk2021isaac,
title = {{Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning}},
author = {Viktor Makoviychuk and Lukasz Wawrzyniak and Yunrong Guo and Michelle Lu and Kier Storey and Miles Macklin and David Hoeller and Nikita Rudin and Arthur Allshire and Ankur Handa and Gavriel State},
year = {2021},
eprint = {2108.10470},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2108.10470},
}
RMPs — describe each behaviour (reach, avoid obstacles, joint limits) as an acceleration policy plus a Riemannian metric, then combine them geometrically; the basis of later RMPflow and geometric fabrics controllers.RMP：把每种行为（到达、避障、关节限位）表示为加速度策略加黎曼度量，再按几何方式组合；是后来 RMPflow 与几何织物（geometric fabrics）控制器的基础。
@misc{ratliff2018riemannian,
title = {{Riemannian Motion Policies}},
author = {Nathan D. Ratliff and Jan Issac and Daniel Kappler and Stan Birchfield and Dieter Fox},
year = {2018},
eprint = {1801.02854},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/1801.02854},
}