📡 Embodied AI Radar具身智能雷达

Labs机构 › Carnegie Mellon University卡内基梅隆大学

Carnegie Mellon University · embodied AI & robot learning卡内基梅隆大学 · 具身智能与机器人学习

20 curated papers, code, datasets and hardware from Carnegie Mellon University (12 with code), with one-line takeaways.20 个来自卡内基梅隆大学的精选论文、代码、数据集与硬件(12 个附代码),均配一句话要点。

Labs are matched from each entry's affiliation field by simple name rules (data/labs.yaml); joint work appears under every lab named. Only curated entries are listed, because daily arXiv papers carry no affiliations.机构由每个条目的机构字段按名称规则匹配(data/labs.yaml),合作工作会出现在所有署名机构下。仅列出精选条目,因为每日 arXiv 论文不含机构信息。

By topic:按主题: VLA & Robot Foundation ModelsVLA 与机器人基础模型 (1) · Manipulation & Dexterous Hands操作与灵巧手 (6) · Tactile & Visuotactile Sensing触觉与视触觉感知 (3) · Locomotion & Humanoids运动控制与人形机器人 (7) · Navigation & Embodied Agents导航与具身智能体 (3) · Perception, SLAM & 3D Mapping感知、SLAM 与三维建图 (3) · Robot Learning: RL & Imitation机器人学习:强化学习与模仿学习 (1) · Sim-to-Real & Simulators仿真与 Sim-to-Real (3) · Datasets & Benchmarks数据集与基准 (2) · Hardware & Open-Source Robots硬件与开源机器人 (4)

All entries全部条目 (20)

Hardware硬件 2023 · CMU ★

LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning

A 16-DoF dexterous hand buildable for about $2,000 from off-the-shelf parts.16 自由度灵巧手,用现成零件约 2000 美元即可搭建。

BibTeX
@misc{shaw2023leap,
  title = {{LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning}},
  author = {Kenneth Shaw and Ananye Agarwal and Deepak Pathak},
  year = {2023},
  eprint = {2309.06440},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2309.06440},
}
Paper论文 2025 · CMU, NVIDIA

ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills

Learns a residual "delta action" model from real rollouts to fix sim physics, enabling agile athletic humanoid motions.从真实轨迹学习残差"增量动作"模型来修正仿真物理,实现敏捷的人形运动技能。

BibTeX
@misc{he2025asap,
  title = {{ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills}},
  author = {Tairan He and Jiawei Gao and Wenli Xiao and Yuanhang Zhang and Zi Wang and Jiashun Wang and Zhengyi Luo and Guanqi He and Nikhil Sobanbab and Chaoyi Pan and Zeji Yi and Guannan Qu and Kris Kitani and Jessica Hodgins and Linxi "Jim" Fan and Yuke Zhu and Changliu Liu and Guanya Shi},
  year = {2025},
  eprint = {2502.01143},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2502.01143},
}
Locomotion & Humanoids运动控制与人形机器人Humanoid人形sim2realhumanoidrldelta-action
Paper论文 2025 · Stanford, Columbia, CMU, NVIDIA CoRL 2025

DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation

DexUMI — an exoskeleton matched to the robot hand's fingertip workspace, plus inpainting that swaps the human hand for the robot's; reports 86% average success (Inspire, XHand) and ~3.2× faster collection than teleop.DexUMI:按目标机器人手的指尖工作空间设计的可穿戴手部外骨骼,再用视频修补把画面中的人手换成机器人手;论文报告在 Inspire 与 XHand 上平均成功率 86%,采集效率约为遥操作的 3.2 倍。

BibTeX
@misc{xu2025dexumi,
  title = {{DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation}},
  author = {Mengda Xu and Han Zhang and Yifan Hou and Zhenjia Xu and Linxi Fan and Manuela Veloso and Shuran Song},
  year = {2025},
  eprint = {2505.21864},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2505.21864},
}
Hardware & Open-Source Robots硬件与开源机器人Manipulation & Dexterous Hands操作与灵巧手Dexterous hand灵巧手realexoskeletondata-collectiondexterous-handinpainting
Benchmark基准 2025 · UC Berkeley · PKU · BIGAI · Stanford · CMU and others RSS 2025

RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning

MetaSim puts many simulators (Isaac Sim/Gym, MuJoCo, Genesis, SAPIEN…) behind one interface; adds a large synthetic dataset migrated from public benchmarks and unified IL/RL benchmarks.MetaSim 用统一接口封装多种仿真器(Isaac Sim/Gym、MuJoCo、Genesis、SAPIEN、PyBullet 等);提供由公开基准迁移而来的大规模合成数据集,以及按泛化层级划分的统一模仿学习/强化学习基准。

BibTeX
@misc{geng2025roboverse,
  title = {{RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning}},
  author = {Haoran Geng and Feishi Wang and Songlin Wei and Yuyang Li and Bangjun Wang and Boshi An and Charlie Tianyue Cheng and Haozhe Lou and Peihao Li and Yen-Jen Wang and Yutong Liang and Dylan Goetting and Chaoyi Xu and Haozhe Chen and Yuxi Qian and Yiran Geng and Jiageng Mao and Weikang Wan and Mingtong Zhang and Jiangran Lyu and Siheng Zhao and Jiazhao Zhang and Jialiang Zhang and Chengyang Zhao and Haoran Lu and Yufei Ding and Ran Gong and Yuran Wang and Yuxuan Kuang and Ruihai Wu and Baoxiong Jia and Carlo Sferrazza and Hao Dong and Siyuan Huang and Yue Wang and Jitendra Malik and Pieter Abbeel},
  year = {2025},
  eprint = {2504.18904},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2504.18904},
}
Paper论文 2024 · CMU, ETH Zurich RSS 2024

Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion

ABS pairs an agile RL locomotion policy with a learned reach-avoid value network and a recovery policy that takes over near obstacles, so a quadruped can run fast without crashing.ABS 将敏捷的强化学习运动策略与学习的可达-规避价值网络及恢复策略结合,接近障碍时由恢复策略接管,让四足机器人高速奔跑而不碰撞。

BibTeX
@misc{he2024agile,
  title = {{Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion}},
  author = {Tairan He and Chong Zhang and Wenli Xiao and Guanqi He and Changliu Liu and Guanya Shi},
  year = {2024},
  eprint = {2401.17583},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2401.17583},
}
Hardware硬件 2024 · CMU CoRL 2024

Bimanual Dexterity for Complex Tasks

Bidex — a portable, low-latency bimanual dexterous teleop rig (mocap gloves + teacher arms) for 50+ DoF robots; reports better data, collected faster, than Vision Pro or SteamVR teleop.Bidex:便携、低延迟的双臂灵巧手遥操作系统(动捕手套 + 示教臂),可控制 50+ 自由度的机器人;论文报告其采集的数据质量和速度均优于 Vision Pro 与 SteamVR 遥操作。

BibTeX
@misc{shaw2024bimanual,
  title = {{Bimanual Dexterity for Complex Tasks}},
  author = {Kenneth Shaw and Yulong Li and Jiahui Yang and Mohan Kumar Srirama and Ray Liu and Haoyu Xiong and Russell Mendonca and Deepak Pathak},
  year = {2024},
  eprint = {2411.13677},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2411.13677},
}
Hardware & Open-Source Robots硬件与开源机器人Manipulation & Dexterous Hands操作与灵巧手Robot arm机械臂Dexterous hand灵巧手teleoperationbimanualhandmocap-glovelow-cost
Paper论文 2024 · NVIDIA, CMU ICRA 2025

HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots

Distills one policy that supports several humanoid control modes (root velocity, joint angles, keypoint tracking) instead of training a separate controller per mode.蒸馏出单一策略,支持多种人形控制模式(根部速度、关节角、关键点跟踪),不必为每种模式各训练一个控制器。

BibTeX
@misc{he2024hover,
  title = {{HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots}},
  author = {Tairan He and Wenli Xiao and Toru Lin and Zhengyi Luo and Zhenjia Xu and Zhenyu Jiang and Jan Kautz and Changliu Liu and Guanya Shi and Xiaolong Wang and Linxi Fan and Yuke Zhu},
  year = {2024},
  eprint = {2410.21229},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2410.21229},
}
Locomotion & Humanoids运动控制与人形机器人Humanoid人形humanoidwhole-bodyrldistillation
Paper论文 2024 · CMU

Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation

H2O — an RL tracking policy lets an RGB camera stream a person's whole-body motion onto a humanoid in real time.H2O:强化学习跟踪策略借助 RGB 相机,把人的全身动作实时映射到人形机器人。

BibTeX
@misc{he2024learning,
  title = {{Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation}},
  author = {Tairan He and Zhengyi Luo and Wenli Xiao and Chong Zhang and Kris Kitani and Changliu Liu and Guanya Shi},
  year = {2024},
  eprint = {2403.04436},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2403.04436},
}
Locomotion & Humanoids运动控制与人形机器人Humanoid人形sim2realhumanoidteleoperation
Paper论文 2024 · UC Berkeley, Stanford, CMU

Octo: An Open-Source Generalist Robot Policy

A small open diffusion-head Transformer pretrained on Open X-Embodiment that fine-tunes quickly to new robots and sensors.在 Open X-Embodiment 上预训练的开源扩散头 Transformer 策略,可快速微调到新机器人和新传感器。

BibTeX
@misc{team2024octo,
  title = {{Octo: An Open-Source Generalist Robot Policy}},
  author = {Octo Model Team and Dibya Ghosh and Homer Walke and Karl Pertsch and Kevin Black and Oier Mees and Sudeep Dasari and Joey Hejna and Tobias Kreiman and Charles Xu and Jianlan Luo and You Liang Tan and Lawrence Yunliang Chen and Pannag Sanketi and Quan Vuong and Ted Xiao and Dorsa Sadigh and Chelsea Finn and Sergey Levine},
  year = {2024},
  eprint = {2405.12213},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2405.12213},
}
VLA & Robot Foundation ModelsVLA 与机器人基础模型realarmdiffusioncross-embodimentopen-weights
Paper论文 2024 · CMU CoRL 2024

OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning

One RL whole-body controller that a humanoid can follow from a VR headset, verbal commands or an RGB camera, and that also runs learned autonomous policies.一个强化学习全身控制器:人形机器人可跟随 VR 头显、语音指令或 RGB 相机的输入,也可执行学习得到的自主策略。

BibTeX
@misc{he2024omnih,
  title = {{OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning}},
  author = {Tairan He and Zhengyi Luo and Xialin He and Wenli Xiao and Chong Zhang and Weinan Zhang and Kris Kitani and Changliu Liu and Guanya Shi},
  year = {2024},
  eprint = {2406.08858},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2406.08858},
}
Locomotion & Humanoids运动控制与人形机器人Dexterous hand灵巧手Humanoid人形humanoidteleoperationwhole-bodyrlsim-to-real
Paper论文 2023 · CMU

Extreme Parkour with Legged Robots

An end-to-end depth-camera policy makes a small quadruped do long jumps, high jumps and handstands.端到端深度相机策略让小型四足机器人完成跳远、跳高和倒立。

BibTeX
@misc{cheng2023extreme,
  title = {{Extreme Parkour with Legged Robots}},
  author = {Xuxin Cheng and Kexin Shi and Ananye Agarwal and Deepak Pathak},
  year = {2023},
  eprint = {2309.14341},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2309.14341},
}
Locomotion & Humanoids运动控制与人形机器人Quadruped / legged四足 / 足式sim2realquadrupedrlvision
Paper论文 2023 · Meta AI, CMU Science Robotics 2024

Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation

NeuralFeels fuses vision and touch from a multi-fingered hand into an online neural field, reconstructing an object's shape and tracking its pose while the hand manipulates it.NeuralFeels 将多指手的视觉与触觉在线融合为神经场,在操作过程中同时重建物体形状并跟踪其位姿。

BibTeX
@misc{suresh2023neural,
  title = {{Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation}},
  author = {Sudharshan Suresh and Haozhi Qi and Tingfan Wu and Taosha Fan and Luis Pineda and Mike Lambeta and Jitendra Malik and Mrinal Kalakrishnan and Roberto Calandra and Michael Kaess and Joseph Ortiz and Mustafa Mukadam},
  year = {2023},
  eprint = {2312.13469},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2312.13469},
}
Paper论文 2021 · UC Berkeley, CMU, Meta AI

RMA: Rapid Motor Adaptation for Legged Robots

An adaptation module infers environment parameters online, so one policy handles new terrains and payloads in real time.自适应模块在线估计环境参数,使同一策略实时适应新地形和负载。

BibTeX
@misc{kumar2021rma,
  title = {{RMA: Rapid Motor Adaptation for Legged Robots}},
  author = {Ashish Kumar and Zipeng Fu and Deepak Pathak and Jitendra Malik},
  year = {2021},
  eprint = {2107.04034},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/2107.04034},
}
Locomotion & Humanoids运动控制与人形机器人Quadruped / legged四足 / 足式sim2realquadrupedrladaptation
Paper论文 2021 · CMU, Meta AI

ReSkin: versatile, replaceable, lasting tactile skins

A thin magnetic-elastomer skin read by magnetometers; cheap to make and swap, with learned calibration that carries over to new skins.由磁力计读取的薄磁性弹性体皮肤,制作和更换成本低,学习得到的标定可迁移到新皮肤上。

BibTeX
@misc{bhirangi2021reskin,
  title = {{ReSkin: versatile, replaceable, lasting tactile skins}},
  author = {Raunaq Bhirangi and Tess Hellebrekers and Carmel Majidi and Abhinav Gupta},
  year = {2021},
  eprint = {2111.00071},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2111.00071},
}
Simulator仿真器 2021 · CMU

Taxim: An Example-based Simulation Model for GelSight Tactile Sensors

Simulates GelSight images from calibration examples (optical lookup + linear elastic deformation) instead of physics-based rendering, so tactile data can be generated quickly.用标定样本(光学查找表 + 线弹性形变)而不是基于物理的渲染来模拟 GelSight 图像,可快速生成触觉数据。

BibTeX
@misc{si2021taxim,
  title = {{Taxim: An Example-based Simulation Model for GelSight Tactile Sensors}},
  author = {Zilin Si and Wenzhen Yuan},
  year = {2021},
  eprint = {2109.04027},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2109.04027},
}
Paper论文 2020 · CMU, Meta AI ICLR 2020

Learning to Explore using Active Neural SLAM

Splits exploration into a learned neural SLAM module, a global policy that chooses long-term goals and a local policy that reaches them, which is far more sample-efficient than end-to-end RL.将探索拆为可学习的神经 SLAM 模块、选择长期目标的全局策略和到达目标的局部策略,比端到端强化学习样本效率高得多。

BibTeX
@misc{chaplot2020learning,
  title = {{Learning to Explore using Active Neural SLAM}},
  author = {Devendra Singh Chaplot and Dhiraj Gandhi and Saurabh Gupta and Abhinav Gupta and Ruslan Salakhutdinov},
  year = {2020},
  eprint = {2004.05155},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url = {https://arxiv.org/abs/2004.05155},
}
Paper论文 2020 · CMU, Meta AI NeurIPS 2020

Object Goal Navigation using Goal-Oriented Semantic Exploration

SemExp builds an episodic semantic map and learns a policy that picks long-term goals on it, a modular recipe that became a standard ObjectNav baseline in Habitat.SemExp 构建情景语义地图,并学习在地图上选择长期目标的策略;这一模块化方案成为 Habitat 中 ObjectNav 的常用基线。

BibTeX
@misc{chaplot2020object,
  title = {{Object Goal Navigation using Goal-Oriented Semantic Exploration}},
  author = {Devendra Singh Chaplot and Dhiraj Gandhi and Abhinav Gupta and Ruslan Salakhutdinov},
  year = {2020},
  eprint = {2007.00643},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url = {https://arxiv.org/abs/2007.00643},
}
Paper论文 2020 · Carnegie Mellon University CoRL 2020

SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation

Open-source Gym-style benchmark for manipulating deformable objects such as cloth, rope and liquids; shows that high-dimensional, partially observable states make these tasks hard for standard RL.开源、Gym 接口的可变形物体操作基准(布料、绳子、液体等);实验表明高维且部分可观测的状态让这类任务对常规强化学习方法很困难。

BibTeX
@misc{lin2020softgym,
  title = {{SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation}},
  author = {Xingyu Lin and Yufei Wang and Jake Olkin and David Held},
  year = {2020},
  eprint = {2011.07215},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2011.07215},
}
Dataset数据集 2015 · Yale · CMU · UC Berkeley

Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols

The YCB object set — 73 everyday objects with scans, textured meshes and simulation models, plus benchmarking protocols; widely used in grasping and pose-estimation work.YCB 物体集:73 个日常物体,附扫描数据、带纹理网格与仿真模型,以及基准测试规程;被抓取与位姿估计研究广泛使用。

BibTeX
@misc{calli2015benchmarking,
  title = {{Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols}},
  author = {Berk Calli and Aaron Walsman and Arjun Singh and Siddhartha Srinivasa and Pieter Abbeel and Aaron M. Dollar},
  year = {2015},
  eprint = {1502.03143},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/1502.03143},
}
Paper论文 2010 · CMU

A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

DAgger — query the expert on the states the learner itself visits, fixing the compounding errors of plain behavior cloning.DAgger:在学习者自己访问到的状态上向专家查询标注,解决单纯行为克隆的误差累积问题。

BibTeX
@misc{ross2010reduction,
  title = {{A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning}},
  author = {Stephane Ross and Geoffrey J. Gordon and J. Andrew Bagnell},
  year = {2010},
  eprint = {1011.0686},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/1011.0686},
}