📡 Embodied AI Radar具身智能雷达

Start here: a reading path入门路线：从这里开始读

10 steps, 41 papers, code bases and robots, picked from the curated index for newcomers. Each links to a page with the bilingual takeaway, code, BibTeX and related work. Skip any step you already know.为新手从精选库中挑选的 10 个阶段、41 篇论文/代码/硬件。每一项都链接到详情页（中英文要点、代码、BibTeX 与相关工作）。已经熟悉的部分可以跳过。

1 · Robot learning basics: RL and imitation1 · 机器人学习基础：强化学习与模仿学习

Almost every later paper builds on these: an on-policy and an off-policy RL workhorse, the classic fix for compounding errors in behavior cloning, and a careful study of what matters when learning from human demos.后面几乎所有工作都建立在这些之上：一个 on-policy 和一个 off-policy 的常用强化学习算法、行为克隆误差累积问题的经典解法，以及一项关于从人类演示学习时哪些因素最重要的系统研究。

  1. Proximal Policy Optimization Algorithms · 2017 · OpenAI
    The clipped policy-gradient method behind most GPU-parallel sim-to-real locomotion and dexterity training.采用截断目标的策略梯度方法，是大多数 GPU 并行仿真训练（运动控制、灵巧操作）背后的算法。
  2. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor · 2018 · UC Berkeley
    Off-policy RL with an entropy bonus; sample-efficient enough to become the default for learning directly on real robots.带熵奖励的离策略强化学习；样本效率较高，成为真机上直接学习的常用默认算法。
  3. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning · 2010 · CMU
    DAgger — query the expert on the states the learner itself visits, fixing the compounding errors of plain behavior cloning.DAgger：在学习者自己访问到的状态上向专家查询标注，解决单纯行为克隆的误差累积问题。
  4. What Matters in Learning from Offline Human Demonstrations for Robot Manipulation · 2021 · Stanford
    robomimic — a controlled study of imitation and offline RL on human-collected manipulation demos (demo quality, observation choices, history-aware BC), released with the framework and datasets.robomimic：在人类采集的操作示教数据上系统比较模仿学习与离线强化学习（示教质量、观测选择、带历史的行为克隆），并开源框架与数据集。

2 · Visuomotor policies for manipulation2 · 操作任务的视觉运动策略

The policy classes most labs start from today (Diffusion Policy, ACT and its 3D variant), plus UMI, which shows how to collect demos without a robot.如今多数实验室的起点策略（Diffusion Policy、ACT 及其 3D 版本），以及 UMI——不用机器人也能采集演示数据的方法。

  1. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion · 2023 · Columbia, TRI, MIT
    Generating action sequences by denoising handles multimodal demos far better than regression; now a standard IL baseline.以去噪方式生成动作序列，比回归更能处理多模态演示，已成为模仿学习标准基线。
  2. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware · 2023 · Stanford
    ALOHA + ACT — a ~$20k bimanual teleop rig and an action-chunking Transformer that learns precise tasks from ~50 demos.ALOHA + ACT：约 2 万美元的双臂遥操作平台，加上动作分块 Transformer，约 50 条演示即可学会精细任务。
  3. 3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations · 2024 · Shanghai Qi Zhi, Tsinghua
    Conditions a diffusion policy on a compact point-cloud encoding; more data-efficient and robust to viewpoint than images.用紧凑点云编码作为扩散策略条件，比图像输入更省数据、对视角更鲁棒。
  4. Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots · 2024 · Stanford, Columbia, TRI
    A handheld gripper with a GoPro collects demos anywhere; policies transfer directly to robot arms.手持夹爪加 GoPro 可随处采集演示，训练出的策略可直接迁移到机械臂。

3 · Data, benchmarks and tooling3 · 数据、基准与工具链

The large shared datasets behind generalist policies, a standard simulated benchmark for comparing methods, and LeRobot, the easiest open stack to train and run policies with.通用策略背后的大规模共享数据集、用于横向比较方法的标准仿真基准，以及 LeRobot——训练和部署策略最容易上手的开源工具链。

  1. Open X-Embodiment: Robotic Learning Datasets and RT-X Models · 2023 · Open X-Embodiment Collaboration
    1M+ trajectories from 22 robot types pooled by 30+ labs; RT-X models trained on it beat single-robot baselines.30 多个实验室汇集 22 种机器人的 100 多万条轨迹；在其上训练的 RT-X 优于单机器人基线。
  2. DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset · 2024 · DROID Collaboration
    76k teleoperated Franka trajectories collected in hundreds of real scenes across many institutions.由多家机构在数百个真实场景中采集的 7.6 万条 Franka 遥操作轨迹。
  3. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning · 2023 · UT Austin
    130 language-conditioned tasks in four suites; now the most common simulation benchmark for VLA papers.四个套件共 130 个语言条件任务，现已成为 VLA 论文最常用的仿真基准。
  4. LeRobot · 2024 · Hugging Face
    PyTorch library with datasets, pretrained policies (ACT, Diffusion, SmolVLA) and drivers for low-cost arms.PyTorch 机器人学习库，提供数据集、预训练策略（ACT、Diffusion、SmolVLA）与低成本机械臂驱动。

4 · Vision-language-action models4 · 视觉-语言-动作模型（VLA）

Read in order: RT-2 introduced the idea, OpenVLA made it open, π0 added a flow-matching action expert, and GR00T N1 / SmolVLA are open models you can fine-tune yourself.建议按顺序读：RT-2 提出了这一思路，OpenVLA 将其开源，π0 引入了 flow matching 动作专家，GR00T N1 和 SmolVLA 是可以自己微调的开源模型。

  1. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control · 2023 · Google DeepMind
    Coined "VLA" — co-fine-tuning a web-scale VLM to output action tokens transfers semantic knowledge to robot control.提出 "VLA" 概念：把网络规模的视觉语言模型联合微调为输出动作 token，把语义知识迁移到机器人控制。
  2. OpenVLA: An Open-Source Vision-Language-Action Model · 2024 · Stanford, UC Berkeley, TRI
    A 7B open VLA built on Llama 2 + DINOv2/SigLIP; the de-facto open baseline for VLA research and LoRA fine-tuning.基于 Llama 2 与 DINOv2/SigLIP 的 7B 开源 VLA，是 VLA 研究与 LoRA 微调的事实开源基线。
  3. π0: A Vision-Language-Action Flow Model for General Robot Control · 2024 · Physical Intelligence
    A VLM backbone with a flow-matching action expert, trained across many robots, doing long dexterous tasks like laundry folding.VLM 主干加流匹配动作专家，跨多种机器人训练，能完成叠衣服等长时程灵巧任务。
  4. GR00T N1: An Open Foundation Model for Generalist Humanoid Robots · 2025 · NVIDIA
    A dual-system (VLM "slow" + diffusion "fast") humanoid policy trained on real, human-video and synthetic data.双系统（VLM 慢思考 + 扩散快执行）人形机器人策略，用真实、人类视频与合成数据训练。
  5. SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics · 2025 · Hugging Face
    A ~450M-parameter VLA trained on community LeRobot data that runs on consumer GPUs.约 4.5 亿参数的 VLA，用社区 LeRobot 数据训练，可在消费级 GPU 上运行。

5 · Simulation and sim-to-real5 · 仿真与 sim-to-real

A physics engine widely used in robot-learning research, two GPU-parallel frameworks for training at scale, and the domain-randomization paper that popularized randomizing the simulator for sim-to-real transfer.机器人学习研究中广泛使用的物理引擎、两个用于大规模训练的 GPU 并行框架，以及让“随机化仿真器参数以实现 sim-to-real 迁移”广为流行的域随机化论文。

  1. MuJoCo · 2012 · Google DeepMind
    Fast, accurate contact physics; the default simulator for robot learning research, open-sourced by DeepMind.快速准确的接触物理仿真，机器人学习研究的默认仿真器，由 DeepMind 开源。
  2. Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning · 2025 · NVIDIA
    The reference paper for Isaac Lab, Orbit's successor — GPU-parallel physics, rendering and sensor simulation with ready-made robots and tasks for RL and imitation.Isaac Lab（Orbit 的后继）的正式论文：GPU 并行的物理、渲染与传感器仿真，内置机器人与任务，面向强化学习和模仿学习。
  3. ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI · 2024 · UC San Diego
    GPU-parallel simulation and rendering with many manipulation tasks; fast visual RL and data generation.GPU 并行仿真与渲染，内置大量操作任务，适合快速视觉强化学习与数据生成。
  4. Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World · 2017 · OpenAI
    Randomizing textures, lighting and cameras in sim makes vision models work on real images with no real data.在仿真中随机化纹理、光照和相机，使视觉模型无需真实数据即可在真实图像上工作。

6 · Legged locomotion and humanoids6 · 足式运动与人形机器人

The massively parallel RL recipe, rapid adaptation to new terrain, a learned transformer controller for real-world humanoid walking, and a sim-to-real alignment method for agile humanoid skills. Unitree RL Gym is the practical starting code.大规模并行强化学习的训练范式、对新地形的快速自适应、在真实人形机器人上行走的学习型 Transformer 控制器，以及面向敏捷人形技能的 sim-to-real 对齐方法。Unitree RL Gym 是实际上手的起点代码。

  1. Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning · 2021 · ETH Zurich, NVIDIA
    Thousands of parallel GPU environments train a quadruped walking policy in minutes; legged_gym became a standard starter.数千个 GPU 并行环境几分钟就能训练出四足行走策略；legged_gym 成为标准入门框架。
  2. RMA: Rapid Motor Adaptation for Legged Robots · 2021 · UC Berkeley, CMU, Meta AI
    An adaptation module infers environment parameters online, so one policy handles new terrains and payloads in real time.自适应模块在线估计环境参数，使同一策略实时适应新地形和负载。
  3. Real-World Humanoid Locomotion with Reinforcement Learning · 2023 · UC Berkeley
    A causal Transformer policy trained in simulation walks a full-size humanoid outdoors, zero-shot.在仿真中训练的因果 Transformer 策略零样本驱动全尺寸人形机器人在户外行走。
  4. ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills · 2025 · CMU, NVIDIA
    Learns a residual "delta action" model from real rollouts to fix sim physics, enabling agile athletic humanoid motions.从真实轨迹学习残差"增量动作"模型来修正仿真物理，实现敏捷的人形运动技能。
  5. Unitree RL Gym · 2024 · Unitree
    Unitree's official starting point for RL locomotion on Go2, H1 and G1 — train in Isaac Gym, check sim-to-sim in MuJoCo, then deploy to the real robot.宇树官方的强化学习运动控制入门仓库，支持 Go2、H1、G1：在 Isaac Gym 中训练，在 MuJoCo 中做 sim2sim 验证，再部署到真机。

7 · World models7 · 世界模型

From latent-imagination RL (DreamerV3) to learned video simulators (UniSim), self-supervised video models used for planning (V-JEPA 2) and a world-foundation-model platform for physical AI (Cosmos).从隐空间想象式强化学习（DreamerV3），到学习得到的视频仿真器（UniSim）、用于规划的自监督视频模型（V-JEPA 2），再到面向物理 AI 的世界基础模型平台（Cosmos）。

  1. Mastering Diverse Domains through World Models · 2023 · Google DeepMind
    DreamerV3 — one set of hyperparameters works across 150+ tasks, including collecting diamonds in Minecraft from scratch.DreamerV3：一套超参数适用于 150 多个任务，包括从零开始在《我的世界》中挖到钻石。
  2. Learning Interactive Real-World Simulators · 2023 · UC Berkeley, Google DeepMind
    UniSim — an action-conditioned video model used as a simulator to train policies that transfer to the real world.UniSim：以动作为条件的视频生成模型充当仿真器，训练出的策略可迁移到真实世界。
  3. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning · 2025 · Meta AI
    A self-supervised video encoder, post-trained with a little robot data, plans zero-shot pick-and-place by latent prediction.自监督视频编码器经少量机器人数据后训练，通过潜空间预测实现零样本抓放规划。
  4. Cosmos World Foundation Model Platform for Physical AI · 2025 · NVIDIA
    Open video world foundation models plus tokenizers and data pipelines for generating physical-AI training data.开放的视频世界基础模型及分词器与数据管线，用于生成物理 AI 训练数据。

8 · Language, planning and navigation8 · 语言、规划与导航

How large language models were first connected to robot skills (SayCan, Code as Policies), language-indexed maps for navigation, and a general visual navigation foundation model.大语言模型最早如何与机器人技能结合（SayCan、Code as Policies），可用语言索引的导航地图，以及一个通用的视觉导航基础模型。

  1. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances · 2022 · Google
    SayCan — an LLM proposes steps and learned value functions score what the robot can actually do right now.SayCan：大模型提出步骤，学习到的价值函数评估机器人当下实际能做什么。
  2. Code as Policies: Language Model Programs for Embodied Control · 2022 · Google
    LLMs write Python that calls perception and control APIs, turning instructions into reactive robot programs.大模型编写调用感知与控制 API 的 Python 代码，把指令变成可响应的机器人程序。
  3. Visual Language Maps for Robot Navigation · 2022 · Freiburg, Google
    VLMaps fuses visual-language features into a 3D map so robots can navigate to open-vocabulary spatial goals.VLMaps 将视觉语言特征融合进 3D 地图，使机器人可按开放词汇空间目标导航。
  4. ViNT: A Foundation Model for Visual Navigation · 2023 · UC Berkeley
    A Transformer navigation policy trained on many robots' driving data that adapts to new goal types and robots with light fine-tuning.在多种机器人的导航数据上训练的 Transformer 导航策略，经少量微调即可适配新的目标形式和新机器人。

9 · Perception for robots9 · 机器人感知

Off-the-shelf models that save weeks of work: segmentation and tracking (SAM 2), 6D pose of novel objects (FoundationPose) and 3D reconstruction from a few images (DUSt3R).能省下数周工作量的现成模型：分割与跟踪（SAM 2）、新物体 6D 位姿估计（FoundationPose），以及由少量图像做三维重建（DUSt3R）。

  1. SAM 2: Segment Anything in Images and Videos · 2024 · Meta AI
    Extends SAM to video with a streaming memory, so a mask prompted once is tracked across frames — handy for following objects during manipulation.用流式记忆把 SAM 扩展到视频：一次提示得到的掩码可在后续帧中持续跟踪，适合在操作过程中跟随目标物体。
  2. FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects · 2023 · NVIDIA
    6D pose estimation and tracking of unseen objects from a CAD model or a few reference images, with no per-object training.只需 CAD 模型或少量参考图像即可对未见过的物体做 6D 位姿估计与跟踪，无需逐物体训练。
  3. DUSt3R: Geometric 3D Vision Made Easy · 2023 · NAVER LABS Europe
    Regresses 3D point maps straight from image pairs without known camera intrinsics or poses; started the "3D foundation model" wave.不需要已知相机内参或位姿，直接从图像对回归三维点图，开启了“三维基础模型”这一方向。

10 · Hardware to get your hands dirty10 · 动手用的硬件

Low-cost, open designs you can build or buy: a hobby-grade arm that works with LeRobot, a leader-arm teleoperation kit, a dexterous hand, and the bimanual ALOHA 2 setup.可以自己搭建或购买的低成本开源设计：可配合 LeRobot 使用的入门机械臂、主从臂遥操作套件、灵巧手，以及双臂 ALOHA 2 平台。

  1. SO-ARM100 / SO-101 · 2024 · The Robot Studio, Hugging Face
    An open 3D-printed 6-DoF arm (leader + follower) costing roughly $100–200 per arm; the default LeRobot starter kit.开源 3D 打印 6 自由度机械臂（主从臂），每只约 100–200 美元，是 LeRobot 的默认入门套件。
  2. GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators · 2023 · UC Berkeley
    3D-printed kinematic replicas of robot arms for cheap, intuitive joint-space teleoperation.3D 打印的机械臂运动学复刻件，实现低成本、直观的关节空间遥操作。
  3. LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning · 2023 · CMU
    A 16-DoF dexterous hand buildable for about $2,000 from off-the-shelf parts.16 自由度灵巧手，用现成零件约 2000 美元即可搭建。
  4. ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation · 2024 · Google DeepMind
    Upgraded ALOHA bimanual rig (stronger grippers, gravity compensation, sturdier frame) with open hardware designs and a MuJoCo model for simulation.ALOHA 双臂平台的升级版（更强夹爪、重力补偿、更稳固的框架），开源硬件设计并提供 MuJoCo 仿真模型。

Courses and other lists: 课程与其他资源列表：Learning resources学习资源 · Suggest a change to this path对路线提建议