MIT · embodied AI & robot learning麻省理工学院 · 具身智能与机器人学习
13 curated papers, code, datasets and hardware from MIT (8 with code), with one-line takeaways.13 个来自麻省理工学院的精选论文、代码、数据集与硬件(8 个附代码),均配一句话要点。
Labs are matched from each entry's affiliation field by simple name rules (data/labs.yaml); joint work appears under every lab named. Only curated entries are listed, because daily arXiv papers carry no affiliations.机构由每个条目的机构字段按名称规则匹配(data/labs.yaml),合作工作会出现在所有署名机构下。仅列出精选条目,因为每日 arXiv 论文不含机构信息。
Generating action sequences by denoising handles multimodal demos far better than regression; now a standard IL baseline.以去噪方式生成动作序列,比回归更能处理多模态演示,已成为模仿学习标准基线。
@misc{chi2023diffusion,
title = {{Diffusion Policy: Visuomotor Policy Learning via Action Diffusion}},
author = {Cheng Chi and Zhenjia Xu and Siyuan Feng and Eric Cousineau and Yilun Du and Benjamin Burchfiel and Russ Tedrake and Shuran Song},
year = {2023},
eprint = {2303.04137},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2303.04137},
}
Asks whether joint readings alone suffice for millimetre-precise insertion — trains sim-to-real policies with no camera, force or tactile input.探究仅靠关节读数能否完成毫米级精密插装:训练不用相机、力或触觉输入的 sim-to-real 策略。
A 3D-aware LLM with scene, object and action tokens, aligned with diffusion models so it can imagine goal images and point clouds before planning actions; trained on ~2M 3D-language-action pairs the authors assembled.具备三维感知的大语言模型,引入场景、物体与动作交互 token,并与扩散模型对齐,可在规划动作前“想象”目标图像和点云;训练数据为作者整理的约 200 万条三维-语言-动作样本。
@misc{zhen2024d,
title = {{3D-VLA: A 3D Vision-Language-Action Generative World Model}},
author = {Haoyu Zhen and Xiaowen Qiu and Peihao Chen and Jincheng Yang and Xin Yan and Yilun Du and Yining Hong and Chuang Gan},
year = {2024},
eprint = {2403.09631},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2403.09631},
}
@misc{cheng2024open,
title = {{Open-TeleVision: Teleoperation with Immersive Active Visual Feedback}},
author = {Xuxin Cheng and Jialong Li and Shiqi Yang and Ge Yang and Xiaolong Wang},
year = {2024},
eprint = {2407.01512},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2407.01512},
}
HPT: per-robot stems turn proprioception and vision into a few tokens for one shared transformer trunk, pretrained on 52 datasets (robot, sim, human video); reports 20%+ gains on unseen tasks.HPT:为每种机器人设计“输入头”,把本体感知与视觉映射成短 token 序列,送入共享的 Transformer 主干;在 52 个数据集(真机、仿真、人类视频)上预训练,报告在未见任务上微调提升 20% 以上。
Builds an object-level 3D scene graph labelled by vision-language models, which an LLM planner can query in natural language.用视觉语言模型构建带语义标注的物体级三维场景图,大模型规划器可以用自然语言查询。
@misc{gu2023conceptgraphs,
title = {{ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning}},
author = {Qiao Gu and Alihusein Kuwajerwala and Sacha Morin and Krishna Murthy Jatavallabhula and Bipasha Sen and Aditya Agarwal and Corban Rivera and William Paul and Kirsty Ellis and Rama Chellappa and Chuang Gan and Celso Miguel de Melo and Joshua B. Tenenbaum and Antonio Torralba and Florian Shkurti and Liam Paull},
year = {2023},
eprint = {2309.16650},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2309.16650},
}
UniPi — casts decision making as text-conditioned video generation, then recovers actions from the generated frames with an inverse dynamics model.UniPi:把决策问题转化为文本条件的视频生成,再用逆动力学模型从生成的画面中恢复动作。
@misc{du2023learning,
title = {{Learning Universal Policies via Text-Guided Video Generation}},
author = {Yilun Du and Mengjiao Yang and Bo Dai and Hanjun Dai and Ofir Nachum and Joshua B. Tenenbaum and Dale Schuurmans and Pieter Abbeel},
year = {2023},
eprint = {2302.00111},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2302.00111},
}
@misc{chen2022visual,
title = {{Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes}},
author = {Tao Chen and Megha Tippur and Siyang Wu and Vikash Kumar and Edward Adelson and Pulkit Agrawal},
year = {2022},
eprint = {2211.11744},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2211.11744},
}
Trains one quadruped policy that exposes gait parameters (frequency, foot swing, posture) as commands, so a human can tune behavior at deploy time for new tasks; open sim-to-real stack for the Unitree Go1.训练一个把步态参数(频率、抬脚高度、姿态等)作为指令开放的四足策略,部署时可由人调节行为以适应新任务;附带面向 Unitree Go1 的开源仿真到真实工具链。
@misc{margolis2022walk,
title = {{Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior}},
author = {Gabriel B Margolis and Pulkit Agrawal},
year = {2022},
eprint = {2212.03238},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2212.03238},
}
A slim GelSight-style finger that measures contact shape, force and slip at high resolution, designed to fit robot grippers.纤薄的 GelSight 类触觉手指,可高分辨率测量接触形状、力与滑移,便于装在机器人夹爪上。
@misc{taylor2021gelslim,
title = {{GelSlim 3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger}},
author = {Ian Taylor and Siyuan Dong and Alberto Rodriguez},
year = {2021},
eprint = {2103.12269},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2103.12269},
}
@misc{shan2020lio,
title = {{LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping}},
author = {Tixiao Shan and Brendan Englot and Drew Meyers and Wei Wang and Carlo Ratti and Daniela Rus},
year = {2020},
eprint = {2007.00258},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2007.00258},
}
Convex MPC for quadrupeds — simplified rigid-body dynamics turn ground-reaction-force planning into a small QP solved in real time; the classic model-based baseline behind many open-source quadruped controllers.四足机器人的凸 MPC:用简化刚体动力学把足端地面反作用力规划变成可实时求解的小规模 QP;众多开源四足控制器背后的经典基于模型的基线。
Model-based toolbox for robotics - multibody dynamics with contact simulation, mathematical programming and trajectory optimization; used in the MIT manipulation and underactuated courses.面向机器人的基于模型的工具箱:带接触的多体动力学仿真、数学规划与轨迹优化;MIT 操作与欠驱动课程即使用它。