📡 Embodied AI Radar具身智能雷达

Labs机构 › MIT麻省理工学院

MIT · embodied AI & robot learning麻省理工学院 · 具身智能与机器人学习

13 curated papers, code, datasets and hardware from MIT (8 with code), with one-line takeaways.13 个来自麻省理工学院的精选论文、代码、数据集与硬件(8 个附代码),均配一句话要点。

Labs are matched from each entry's affiliation field by simple name rules (data/labs.yaml); joint work appears under every lab named. Only curated entries are listed, because daily arXiv papers carry no affiliations.机构由每个条目的机构字段按名称规则匹配(data/labs.yaml),合作工作会出现在所有署名机构下。仅列出精选条目,因为每日 arXiv 论文不含机构信息。

By topic:按主题: VLA & Robot Foundation ModelsVLA 与机器人基础模型 (2) · Manipulation & Dexterous Hands操作与灵巧手 (3) · Tactile & Visuotactile Sensing触觉与视触觉感知 (1) · Locomotion & Humanoids运动控制与人形机器人 (3) · World Models世界模型 (2) · Navigation & Embodied Agents导航与具身智能体 (1) · Perception, SLAM & 3D Mapping感知、SLAM 与三维建图 (2) · Planning, Control & Safety规划、控制与安全 (2) · Robot Learning: RL & Imitation机器人学习:强化学习与模仿学习 (1) · Sim-to-Real & Simulators仿真与 Sim-to-Real (2) · Hardware & Open-Source Robots硬件与开源机器人 (2)

Recently added最近加入

All entries全部条目 (13)

Paper论文 2023 · Columbia, TRI, MIT ★

Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

Generating action sequences by denoising handles multimodal demos far better than regression; now a standard IL baseline.以去噪方式生成动作序列,比回归更能处理多模态演示,已成为模仿学习标准基线。

BibTeX
@misc{chi2023diffusion,
  title = {{Diffusion Policy: Visuomotor Policy Learning via Action Diffusion}},
  author = {Cheng Chi and Zhenjia Xu and Siyuan Feng and Eric Cousineau and Yilun Du and Benjamin Burchfiel and Russ Tedrake and Shuran Song},
  year = {2023},
  eprint = {2303.04137},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2303.04137},
}
Paper论文 2026 · MIT new新 2026-10-19

Blind Precise Assembly: Proprioceptive Sim-to-Real Peg-in-Hole for Diverse Parts

Asks whether joint readings alone suffice for millimetre-precise insertion — trains sim-to-real policies with no camera, force or tactile input.探究仅靠关节读数能否完成毫米级精密插装:训练不用相机、力或触觉输入的 sim-to-real 策略。

BibTeX
@misc{bronars2026blind,
  title = {{Blind Precise Assembly: Proprioceptive Sim-to-Real Peg-in-Hole for Diverse Parts}},
  author = {Antonia Bronars and Younghyo Park and Kushal Kedia and Pulkit Agrawal},
  year = {2026},
  eprint = {2610.23849},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2610.23849},
}
Paper论文 2024 · UMass Amherst / MIT ICML 2024

3D-VLA: A 3D Vision-Language-Action Generative World Model

A 3D-aware LLM with scene, object and action tokens, aligned with diffusion models so it can imagine goal images and point clouds before planning actions; trained on ~2M 3D-language-action pairs the authors assembled.具备三维感知的大语言模型,引入场景、物体与动作交互 token,并与扩散模型对齐,可在规划动作前“想象”目标图像和点云;训练数据为作者整理的约 200 万条三维-语言-动作样本。

BibTeX
@misc{zhen2024d,
  title = {{3D-VLA: A 3D Vision-Language-Action Generative World Model}},
  author = {Haoyu Zhen and Xiaowen Qiu and Peihao Chen and Jincheng Yang and Xin Yan and Yilun Du and Yining Hong and Chuang Gan},
  year = {2024},
  eprint = {2403.09631},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url = {https://arxiv.org/abs/2403.09631},
}
Hardware硬件 2024 · UC San Diego, MIT

Open-TeleVision: Teleoperation with Immersive Active Visual Feedback

VR teleoperation with an actively steered stereo camera, used to collect humanoid manipulation data.带主动转向立体相机的 VR 遥操作系统,用于采集人形机器人操作数据。

BibTeX
@misc{cheng2024open,
  title = {{Open-TeleVision: Teleoperation with Immersive Active Visual Feedback}},
  author = {Xuxin Cheng and Jialong Li and Shiqi Yang and Ge Yang and Xiaolong Wang},
  year = {2024},
  eprint = {2407.01512},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2407.01512},
}
Paper论文 2024 · MIT NeurIPS 2024

Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers

HPT: per-robot stems turn proprioception and vision into a few tokens for one shared transformer trunk, pretrained on 52 datasets (robot, sim, human video); reports 20%+ gains on unseen tasks.HPT:为每种机器人设计“输入头”,把本体感知与视觉映射成短 token 序列,送入共享的 Transformer 主干;在 52 个数据集(真机、仿真、人类视频)上预训练,报告在未见任务上微调提升 20% 以上。

BibTeX
@misc{wang2024scaling,
  title = {{Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers}},
  author = {Lirui Wang and Xinlei Chen and Jialiang Zhao and Kaiming He},
  year = {2024},
  eprint = {2409.20537},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2409.20537},
}
Paper论文 2023 · University of Toronto · MIT · Mila

ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning

Builds an object-level 3D scene graph labelled by vision-language models, which an LLM planner can query in natural language.用视觉语言模型构建带语义标注的物体级三维场景图,大模型规划器可以用自然语言查询。

BibTeX
@misc{gu2023conceptgraphs,
  title = {{ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning}},
  author = {Qiao Gu and Alihusein Kuwajerwala and Sacha Morin and Krishna Murthy Jatavallabhula and Bipasha Sen and Aditya Agarwal and Corban Rivera and William Paul and Kirsty Ellis and Rama Chellappa and Chuang Gan and Celso Miguel de Melo and Joshua B. Tenenbaum and Antonio Torralba and Florian Shkurti and Liam Paull},
  year = {2023},
  eprint = {2309.16650},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2309.16650},
}
Paper论文 2023 · MIT, Google NeurIPS 2023

Learning Universal Policies via Text-Guided Video Generation

UniPi — casts decision making as text-conditioned video generation, then recovers actions from the generated frames with an inverse dynamics model.UniPi:把决策问题转化为文本条件的视频生成,再用逆动力学模型从生成的画面中恢复动作。

BibTeX
@misc{du2023learning,
  title = {{Learning Universal Policies via Text-Guided Video Generation}},
  author = {Yilun Du and Mengjiao Yang and Bo Dai and Hanjun Dai and Ofir Nachum and Joshua B. Tenenbaum and Dale Schuurmans and Pieter Abbeel},
  year = {2023},
  eprint = {2302.00111},
  archivePrefix = {arXiv},
  primaryClass = {cs.AI},
  url = {https://arxiv.org/abs/2302.00111},
}
World Models世界模型video-generationplanninglanguage
Paper论文 2022 · MIT

Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes

Reorients unseen objects in-hand using only a depth camera, with a policy trained in simulation.仅凭深度相机即可在手中翻转未见过的物体,策略在仿真中训练。

BibTeX
@misc{chen2022visual,
  title = {{Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes}},
  author = {Tao Chen and Megha Tippur and Siyang Wu and Vikash Kumar and Edward Adelson and Pulkit Agrawal},
  year = {2022},
  eprint = {2211.11744},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2211.11744},
}
Manipulation & Dexterous Hands操作与灵巧手Dexterous hand灵巧手sim2realhandrldepth
Paper论文 2022 · MIT CoRL 2022

Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior

Trains one quadruped policy that exposes gait parameters (frequency, foot swing, posture) as commands, so a human can tune behavior at deploy time for new tasks; open sim-to-real stack for the Unitree Go1.训练一个把步态参数(频率、抬脚高度、姿态等)作为指令开放的四足策略,部署时可由人调节行为以适应新任务;附带面向 Unitree Go1 的开源仿真到真实工具链。

BibTeX
@misc{margolis2022walk,
  title = {{Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior}},
  author = {Gabriel B Margolis and Pulkit Agrawal},
  year = {2022},
  eprint = {2212.03238},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2212.03238},
}
Locomotion & Humanoids运动控制与人形机器人Quadruped / legged四足 / 足式realquadrupedsim2realrlunitree-go1
Paper论文 2021 · MIT

GelSlim 3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger

A slim GelSight-style finger that measures contact shape, force and slip at high resolution, designed to fit robot grippers.纤薄的 GelSight 类触觉手指,可高分辨率测量接触形状、力与滑移,便于装在机器人夹爪上。

BibTeX
@misc{taylor2021gelslim,
  title = {{GelSlim 3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger}},
  author = {Ian Taylor and Siyuan Dong and Alberto Rodriguez},
  year = {2021},
  eprint = {2103.12269},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2103.12269},
}
Paper论文 2020 · MIT

LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping

LiDAR-inertial odometry built on a factor graph, so GPS and loop closures slot in as extra factors.基于因子图的激光-惯性里程计,GPS 与回环可以作为额外因子直接加入。

BibTeX
@misc{shan2020lio,
  title = {{LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping}},
  author = {Tixiao Shan and Brendan Englot and Drew Meyers and Wei Wang and Carlo Ratti and Daniela Rus},
  year = {2020},
  eprint = {2007.00258},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2007.00258},
}
Paper论文 2018 · MIT (Biomimetic Robotics Lab) IROS 2018

Dynamic Locomotion in the MIT Cheetah 3 Through Convex Model-Predictive Control

Convex MPC for quadrupeds — simplified rigid-body dynamics turn ground-reaction-force planning into a small QP solved in real time; the classic model-based baseline behind many open-source quadruped controllers.四足机器人的凸 MPC:用简化刚体动力学把足端地面反作用力规划变成可实时求解的小规模 QP;众多开源四足控制器背后的经典基于模型的基线。

Planning, Control & Safety规划、控制与安全Locomotion & Humanoids运动控制与人形机器人Quadruped / legged四足 / 足式mpcquadrupedconvex-optimizationground-reaction-forces
Code代码 2014 · MIT, Toyota Research Institute

Drake

Model-based toolbox for robotics - multibody dynamics with contact simulation, mathematical programming and trajectory optimization; used in the MIT manipulation and underactuated courses.面向机器人的基于模型的工具箱:带接触的多体动力学仿真、数学规划与轨迹优化;MIT 操作与欠驱动课程即使用它。