📡 Embodied AI Radar具身智能雷达

Labs机构 › Google DeepMind / Google谷歌 DeepMind / 谷歌

Google DeepMind / Google · embodied AI & robot learning谷歌 DeepMind / 谷歌 · 具身智能与机器人学习

32 curated papers, code, datasets and hardware from Google DeepMind / Google (14 with code), with one-line takeaways.32 个来自谷歌 DeepMind / 谷歌的精选论文、代码、数据集与硬件(14 个附代码),均配一句话要点。

Labs are matched from each entry's affiliation field by simple name rules (data/labs.yaml); joint work appears under every lab named. Only curated entries are listed, because daily arXiv papers carry no affiliations.机构由每个条目的机构字段按名称规则匹配(data/labs.yaml),合作工作会出现在所有署名机构下。仅列出精选条目,因为每日 arXiv 论文不含机构信息。

By topic:按主题: VLA & Robot Foundation ModelsVLA 与机器人基础模型 (8) · Manipulation & Dexterous Hands操作与灵巧手 (5) · World Models世界模型 (8) · Navigation & Embodied Agents导航与具身智能体 (5) · Planning, Control & Safety规划、控制与安全 (1) · Robot Learning: RL & Imitation机器人学习:强化学习与模仿学习 (6) · Sim-to-Real & Simulators仿真与 Sim-to-Real (5) · Datasets & Benchmarks数据集与基准 (1) · Hardware & Open-Source Robots硬件与开源机器人 (1)

All entries全部条目 (32)

Paper论文 2023 · Google DeepMind ★

Mastering Diverse Domains through World Models

DreamerV3 — one set of hyperparameters works across 150+ tasks, including collecting diamonds in Minecraft from scratch.DreamerV3:一套超参数适用于 150 多个任务,包括从零开始在《我的世界》中挖到钻石。

BibTeX
@misc{hafner2023mastering,
  title = {{Mastering Diverse Domains through World Models}},
  author = {Danijar Hafner and Jurgis Pasukonis and Jimmy Ba and Timothy Lillicrap},
  year = {2023},
  eprint = {2301.04104},
  archivePrefix = {arXiv},
  primaryClass = {cs.AI},
  url = {https://arxiv.org/abs/2301.04104},
}
World Models世界模型simmodel-based-rl
Paper论文 2023 · Google DeepMind ★

RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control

Coined "VLA" — co-fine-tuning a web-scale VLM to output action tokens transfers semantic knowledge to robot control.提出 "VLA" 概念:把网络规模的视觉语言模型联合微调为输出动作 token,把语义知识迁移到机器人控制。

BibTeX
@misc{brohan2023rt,
  title = {{RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control}},
  author = {Anthony Brohan and Noah Brown and Justice Carbajal and Yevgen Chebotar and Xi Chen and Krzysztof Choromanski and Tianli Ding and Danny Driess and Avinava Dubey and Chelsea Finn and Pete Florence and Chuyuan Fu and Montse Gonzalez Arenas and Keerthana Gopalakrishnan and Kehang Han and Karol Hausman and Alexander Herzog and Jasmine Hsu and Brian Ichter and Alex Irpan and Nikhil Joshi and Ryan Julian and Dmitry Kalashnikov and Yuheng Kuang and Isabel Leal and Lisa Lee and Tsang-Wei Edward Lee and Sergey Levine and Yao Lu and Henryk Michalewski and Igor Mordatch and Karl Pertsch and Kanishka Rao and Krista Reymann and Michael Ryoo and Grecia Salazar and Pannag Sanketi and Pierre Sermanet and Jaspiar Singh and Anikait Singh and Radu Soricut and Huong Tran and Vincent Vanhoucke and Quan Vuong and Ayzaan Wahid and Stefan Welker and Paul Wohlhart and Jialin Wu and Fei Xia and Ted Xiao and others},
  year = {2023},
  eprint = {2307.15818},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2307.15818},
}
Paper论文 2022 · Google ★

Do As I Can, Not As I Say: Grounding Language in Robotic Affordances

SayCan — an LLM proposes steps and learned value functions score what the robot can actually do right now.SayCan:大模型提出步骤,学习到的价值函数评估机器人当下实际能做什么。

BibTeX
@misc{ahn2022do,
  title = {{Do As I Can, Not As I Say: Grounding Language in Robotic Affordances}},
  author = {Michael Ahn and Anthony Brohan and Noah Brown and Yevgen Chebotar and Omar Cortes and Byron David and Chelsea Finn and Chuyuan Fu and Keerthana Gopalakrishnan and Karol Hausman and Alex Herzog and Daniel Ho and Jasmine Hsu and Julian Ibarz and Brian Ichter and Alex Irpan and Eric Jang and Rosario Jauregui Ruano and Kyle Jeffrey and Sally Jesmonth and Nikhil J Joshi and Ryan Julian and Dmitry Kalashnikov and Yuheng Kuang and Kuang-Huei Lee and Sergey Levine and Yao Lu and Linda Luu and Carolina Parada and Peter Pastor and Jornell Quiambao and Kanishka Rao and Jarek Rettinghouse and Diego Reyes and Pierre Sermanet and Nicolas Sievers and Clayton Tan and Alexander Toshev and Vincent Vanhoucke and Fei Xia and Ted Xiao and Peng Xu and Sichun Xu and Mengyuan Yan and Andy Zeng},
  year = {2022},
  eprint = {2204.01691},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2204.01691},
}
Paper论文 2025 · Google DeepMind

Gemini Robotics: Bringing AI into the Physical World

Gemini 2.0-based VLA and embodied-reasoning models for dexterous, general robot control.基于 Gemini 2.0 的 VLA 与具身推理模型,面向灵巧、通用的机器人控制。

BibTeX
@misc{team2025gemini,
  title = {{Gemini Robotics: Bringing AI into the Physical World}},
  author = {Gemini Robotics Team and Saminda Abeyruwan and Joshua Ainslie and Jean-Baptiste Alayrac and Montserrat Gonzalez Arenas and Travis Armstrong and Ashwin Balakrishna and Robert Baruch and Maria Bauza and Michiel Blokzijl and Steven Bohez and Konstantinos Bousmalis and Anthony Brohan and Thomas Buschmann and Arunkumar Byravan and Serkan Cabi and Ken Caluwaerts and Federico Casarini and Oscar Chang and Jose Enrique Chen and Xi Chen and Hao-Tien Lewis Chiang and Krzysztof Choromanski and David D'Ambrosio and Sudeep Dasari and Todor Davchev and Coline Devin and Norman Di Palo and Tianli Ding and Adil Dostmohamed and Danny Driess and Yilun Du and Debidatta Dwibedi and Michael Elabd and Claudio Fantacci and Cody Fong and Erik Frey and Chuyuan Fu and Marissa Giustina and Keerthana Gopalakrishnan and Laura Graesser and Leonard Hasenclever and Nicolas Heess and Brandon Hernaez and Alexander Herzog and R. Alex Hofer and Jan Humplik and Atil Iscen and Mithun George Jacob and Deepali Jain and others},
  year = {2025},
  eprint = {2503.20020},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2503.20020},
}
VLA & Robot Foundation ModelsVLA 与机器人基础模型Robot arm机械臂Dexterous hand灵巧手realbimanualvlm
Paper论文 2025 · Google DeepMind, UC Berkeley

MuJoCo Playground

GPU-accelerated MJX environments for locomotion and manipulation that train in minutes and transfer to real robots.基于 MJX 的 GPU 加速运动与操作环境,几分钟即可训练并迁移到真实机器人。

BibTeX
@misc{zakka2025mujoco,
  title = {{MuJoCo Playground}},
  author = {Kevin Zakka and Baruch Tabanpour and Qiayuan Liao and Mustafa Haiderbhai and Samuel Holt and Jing Yuan Luo and Arthur Allshire and Erik Frey and Koushil Sreenath and Lueder A. Kahrs and Carmelo Sferrazza and Yuval Tassa and Pieter Abbeel},
  year = {2025},
  eprint = {2502.08844},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2502.08844},
}
Sim-to-Real & Simulators仿真与 Sim-to-Realgpu-parallelsim2realopen-source
Simulator仿真器 2025 · Disney Research, Google DeepMind, NVIDIA (Linux Foundation project)

Newton

Open, GPU-accelerated physics engine built on NVIDIA Warp and OpenUSD with multiple solvers for contact-rich robot learning; co-developed by Disney Research, Google DeepMind and NVIDIA, hosted by the Linux Foundation.基于 NVIDIA Warp 和 OpenUSD 的开源 GPU 物理引擎,支持多种求解器,面向接触丰富的机器人学习;由 Disney Research、Google DeepMind 和 NVIDIA 联合开发,由 Linux 基金会托管。

Sim-to-Real & Simulators仿真与 Sim-to-Realphysicsgpu-parallelopen-sourcewarpopenusd
Hardware硬件 2024 · Google DeepMind

ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation

Upgraded ALOHA bimanual rig (stronger grippers, gravity compensation, sturdier frame) with open hardware designs and a MuJoCo model for simulation.ALOHA 双臂平台的升级版(更强夹爪、重力补偿、更稳固的框架),开源硬件设计并提供 MuJoCo 仿真模型。

BibTeX
@misc{team2024aloha,
  title = {{ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation}},
  author = {ALOHA 2 Team and Jorge Aldaco and Travis Armstrong and Robert Baruch and Jeff Bingham and Sanky Chan and Kenneth Draper and Debidatta Dwibedi and Chelsea Finn and Pete Florence and Spencer Goodrich and Wayne Gramlich and Torr Hage and Alexander Herzog and Jonathan Hoech and Thinh Nguyen and Ian Storz and Baruch Tabanpour and Leila Takayama and Jonathan Tompson and Ayzaan Wahid and Ted Wahrburg and Sichun Xu and Sergey Yaroshenko and Kevin Zakka and Tony Z. Zhao},
  year = {2024},
  eprint = {2405.02292},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2405.02292},
}
Benchmark基准 2024 · UC San Diego, Stanford, UC Berkeley, Google DeepMind

Evaluating Real-World Robot Manipulation Policies in Simulation

SimplerEnv — real-to-sim environments whose scores correlate with real-robot results for RT-1/Octo-style policies.SimplerEnv:真实到仿真的评测环境,其得分与 RT-1/Octo 类策略的真机表现相关。

BibTeX
@misc{li2024evaluating,
  title = {{Evaluating Real-World Robot Manipulation Policies in Simulation}},
  author = {Xuanlin Li and Kyle Hsu and Jiayuan Gu and Karl Pertsch and Oier Mees and Homer Rich Walke and Chuyuan Fu and Ishikaa Lunawat and Isabel Sieh and Sean Kirmani and Sergey Levine and Jiajun Wu and Chelsea Finn and Hao Su and Quan Vuong and Ted Xiao},
  year = {2024},
  eprint = {2405.05941},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2405.05941},
}
Paper论文 2024 · Google DeepMind

Genie: Generative Interactive Environments

Learns playable environments and latent actions from unlabeled internet videos alone.仅从无标注的网络视频中学习可交互环境与潜在动作。

BibTeX
@misc{bruce2024genie,
  title = {{Genie: Generative Interactive Environments}},
  author = {Jake Bruce and Michael Dennis and Ashley Edwards and Jack Parker-Holder and Yuge Shi and Edward Hughes and Matthew Lai and Aditi Mavalankar and Richie Steigerwald and Chris Apps and Yusuf Aytar and Sarah Bechtle and Feryal Behbahani and Stephanie Chan and Nicolas Heess and Lucy Gonzalez and Simon Osindero and Sherjil Ozair and Scott Reed and Jingwei Zhang and Konrad Zolna and Jeff Clune and Nando de Freitas and Satinder Singh and Tim Rocktäschel},
  year = {2024},
  eprint = {2402.15391},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/2402.15391},
}
World Models世界模型video-generationlatent-actions
Paper论文 2023 · Stanford, Princeton, UT Austin, Google DeepMind and others IJRR 2024

Foundation Models in Robotics: Applications, Challenges, and the Future

Survey of how pretrained foundation models are used in robot perception, decision-making and control, and what holds them back — scarce robot data, safety and uncertainty, real-time execution.综述预训练基础模型如何用于机器人的感知、决策与控制,以及主要瓶颈:机器人数据稀缺、安全性与不确定性量化、实时执行。

BibTeX
@misc{firoozi2023foundation,
  title = {{Foundation Models in Robotics: Applications, Challenges, and the Future}},
  author = {Roya Firoozi and Johnathan Tucker and Stephen Tian and Anirudha Majumdar and Jiankai Sun and Weiyu Liu and Yuke Zhu and Shuran Song and Ashish Kapoor and Karol Hausman and Brian Ichter and Danny Driess and Jiajun Wu and Cewu Lu and Mac Schwager},
  year = {2023},
  eprint = {2312.07843},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2312.07843},
}
Paper论文 2023 · UC Berkeley, Google DeepMind

Learning Interactive Real-World Simulators

UniSim — an action-conditioned video model used as a simulator to train policies that transfer to the real world.UniSim:以动作为条件的视频生成模型充当仿真器,训练出的策略可迁移到真实世界。

BibTeX
@misc{yang2023learning,
  title = {{Learning Interactive Real-World Simulators}},
  author = {Sherry Yang and Yilun Du and Kamyar Ghasemipour and Jonathan Tompson and Leslie Kaelbling and Dale Schuurmans and Pieter Abbeel},
  year = {2023},
  eprint = {2310.06114},
  archivePrefix = {arXiv},
  primaryClass = {cs.AI},
  url = {https://arxiv.org/abs/2310.06114},
}
World Models世界模型video-generationsimulator
Paper论文 2023 · MIT, Google NeurIPS 2023

Learning Universal Policies via Text-Guided Video Generation

UniPi — casts decision making as text-conditioned video generation, then recovers actions from the generated frames with an inverse dynamics model.UniPi:把决策问题转化为文本条件的视频生成,再用逆动力学模型从生成的画面中恢复动作。

BibTeX
@misc{du2023learning,
  title = {{Learning Universal Policies via Text-Guided Video Generation}},
  author = {Yilun Du and Mengjiao Yang and Bo Dai and Hanjun Dai and Ofir Nachum and Joshua B. Tenenbaum and Dale Schuurmans and Pieter Abbeel},
  year = {2023},
  eprint = {2302.00111},
  archivePrefix = {arXiv},
  primaryClass = {cs.AI},
  url = {https://arxiv.org/abs/2302.00111},
}
World Models世界模型video-generationplanninglanguage
Paper论文 2023 · Google

PaLM-E: An Embodied Multimodal Language Model

Injects continuous sensor embeddings into a 562B language model for embodied reasoning and planning.将连续传感器嵌入注入 5620 亿参数语言模型,用于具身推理与规划。

BibTeX
@misc{driess2023palm,
  title = {{PaLM-E: An Embodied Multimodal Language Model}},
  author = {Danny Driess and Fei Xia and Mehdi S. M. Sajjadi and Corey Lynch and Aakanksha Chowdhery and Brian Ichter and Ayzaan Wahid and Jonathan Tompson and Quan Vuong and Tianhe Yu and Wenlong Huang and Yevgen Chebotar and Pierre Sermanet and Daniel Duckworth and Sergey Levine and Vincent Vanhoucke and Karol Hausman and Marc Toussaint and Klaus Greff and Andy Zeng and Igor Mordatch and Pete Florence},
  year = {2023},
  eprint = {2303.03378},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/2303.03378},
}
Paper论文 2023 · Google DeepMind CoRL 2023

Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions

Discretizes each action dimension and predicts Q-values per dimension autoregressively with a Transformer, plus a conservative regularizer; trained offline on large real robot datasets mixing demos and autonomous data.将每个动作维度离散化,用 Transformer 自回归地逐维预测 Q 值,并加入保守正则;在混合演示与自主采集数据的大规模真实机器人数据集上离线训练。

BibTeX
@misc{chebotar2023q,
  title = {{Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions}},
  author = {Yevgen Chebotar and Quan Vuong and Alex Irpan and Karol Hausman and Fei Xia and Yao Lu and Aviral Kumar and Tianhe Yu and Alexander Herzog and Karl Pertsch and Keerthana Gopalakrishnan and Julian Ibarz and Ofir Nachum and Sumedh Sontakke and Grecia Salazar and Huong T Tran and Jodilyn Peralta and Clayton Tan and Deeksha Manjunath and Jaspiar Singht and Brianna Zitkovich and Tomas Jackson and Kanishka Rao and Chelsea Finn and Sergey Levine},
  year = {2023},
  eprint = {2309.10150},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2309.10150},
}
Paper论文 2022 · DeepMind

A Generalist Agent

Gato — one Transformer with one set of weights plays games, captions images and stacks blocks with a real arm.Gato:同一套权重的 Transformer 既能玩游戏、写图像描述,也能用真实机械臂堆积木。

BibTeX
@misc{reed2022generalist,
  title = {{A Generalist Agent}},
  author = {Scott Reed and Konrad Zolna and Emilio Parisotto and Sergio Gomez Colmenarejo and Alexander Novikov and Gabriel Barth-Maron and Mai Gimenez and Yury Sulsky and Jackie Kay and Jost Tobias Springenberg and Tom Eccles and Jake Bruce and Ali Razavi and Ashley Edwards and Nicolas Heess and Yutian Chen and Raia Hadsell and Oriol Vinyals and Mahyar Bordbar and Nando de Freitas},
  year = {2022},
  eprint = {2205.06175},
  archivePrefix = {arXiv},
  primaryClass = {cs.AI},
  url = {https://arxiv.org/abs/2205.06175},
}
Paper论文 2022 · Google

Code as Policies: Language Model Programs for Embodied Control

LLMs write Python that calls perception and control APIs, turning instructions into reactive robot programs.大模型编写调用感知与控制 API 的 Python 代码,把指令变成可响应的机器人程序。

BibTeX
@misc{liang2022code,
  title = {{Code as Policies: Language Model Programs for Embodied Control}},
  author = {Jacky Liang and Wenlong Huang and Fei Xia and Peng Xu and Karol Hausman and Brian Ichter and Pete Florence and Andy Zeng},
  year = {2022},
  eprint = {2209.07753},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2209.07753},
}
Paper论文 2022 · Google

Inner Monologue: Embodied Reasoning through Planning with Language Models

Feeding success detection and scene descriptions back to the LLM planner lets it replan after failures.将成功检测与场景描述反馈给大模型规划器,使其在失败后重新规划。

BibTeX
@misc{huang2022inner,
  title = {{Inner Monologue: Embodied Reasoning through Planning with Language Models}},
  author = {Wenlong Huang and Fei Xia and Ted Xiao and Harris Chan and Jacky Liang and Pete Florence and Andy Zeng and Jonathan Tompson and Igor Mordatch and Yevgen Chebotar and Pierre Sermanet and Noah Brown and Tomas Jackson and Linda Luu and Sergey Levine and Karol Hausman and Brian Ichter},
  year = {2022},
  eprint = {2207.05608},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2207.05608},
}
Paper论文 2022 · UC Berkeley, Google CoRL 2022

LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action

Chains an LLM (parses landmarks from the instruction), CLIP (finds them in images) and a visual navigation model to follow route instructions on a real robot without language-annotated robot data.将大语言模型(从指令中解析地标)、CLIP(在图像中找到地标)与视觉导航模型串联,无需带语言标注的机器人数据即可在真实机器人上按路线指令导航。

BibTeX
@misc{shah2022lm,
  title = {{LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action}},
  author = {Dhruv Shah and Blazej Osinski and Brian Ichter and Sergey Levine},
  year = {2022},
  eprint = {2207.04429},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2207.04429},
}
Paper论文 2022 · Google DeepMind

Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo

Introduces MuJoCo MPC (MJPC), an interactive tool for real-time predictive control in MuJoCo, and shows that a simple sampling-based planner is a strong baseline for contact-rich behaviours.推出 MuJoCo MPC(MJPC),可在 MuJoCo 中交互式地做实时预测控制,并表明简单的基于采样的规划器在接触丰富的行为上是很强的基线。

BibTeX
@misc{howell2022predictive,
  title = {{Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo}},
  author = {Taylor Howell and Nimrod Gileadi and Saran Tunyasuvunakool and Kevin Zakka and Tom Erez and Yuval Tassa},
  year = {2022},
  eprint = {2212.00541},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2212.00541},
}
Paper论文 2022 · Google

RT-1: Robotics Transformer for Real-World Control at Scale

Tokenized actions plus a Transformer trained on 130k real demos showed that robot policies scale with data diversity.将动作离散化为 token,用 13 万条真实演示训练 Transformer,证明机器人策略会随数据多样性扩展。

BibTeX
@misc{brohan2022rt,
  title = {{RT-1: Robotics Transformer for Real-World Control at Scale}},
  author = {Anthony Brohan and Noah Brown and Justice Carbajal and Yevgen Chebotar and Joseph Dabis and Chelsea Finn and Keerthana Gopalakrishnan and Karol Hausman and Alex Herzog and Jasmine Hsu and Julian Ibarz and Brian Ichter and Alex Irpan and Tomas Jackson and Sally Jesmonth and Nikhil J Joshi and Ryan Julian and Dmitry Kalashnikov and Yuheng Kuang and Isabel Leal and Kuang-Huei Lee and Sergey Levine and Yao Lu and Utsav Malla and Deeksha Manjunath and Igor Mordatch and Ofir Nachum and Carolina Parada and Jodilyn Peralta and Emily Perez and Karl Pertsch and Jornell Quiambao and Kanishka Rao and Michael Ryoo and Grecia Salazar and Pannag Sanketi and Kevin Sayed and Jaspiar Singh and Sumedh Sontakke and Austin Stone and Clayton Tan and Huong Tran and Vincent Vanhoucke and Steve Vega and Quan Vuong and Fei Xia and Ted Xiao and Peng Xu and Sichun Xu and Tianhe Yu and others},
  year = {2022},
  eprint = {2212.06817},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2212.06817},
}
Paper论文 2022 · Freiburg, Google

Visual Language Maps for Robot Navigation

VLMaps fuses visual-language features into a 3D map so robots can navigate to open-vocabulary spatial goals.VLMaps 将视觉语言特征融合进 3D 地图,使机器人可按开放词汇空间目标导航。

BibTeX
@misc{huang2022visual,
  title = {{Visual Language Maps for Robot Navigation}},
  author = {Chenguang Huang and Oier Mees and Andy Zeng and Wolfram Burgard},
  year = {2022},
  eprint = {2210.05714},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2210.05714},
}
Paper论文 2021 · Google NeurIPS 2021 Datasets and Benchmarks

Brax: A Differentiable Physics Engine for Large Scale Rigid Body Simulation

JAX physics engine that runs thousands of environments in parallel on GPU/TPU together with the RL training loop, cutting locomotion training from hours to minutes.基于 JAX 的物理引擎,在 GPU/TPU 上与强化学习训练循环一起并行运行数千个环境,把运动控制训练从数小时缩短到数分钟。

BibTeX
@misc{freeman2021brax,
  title = {{Brax: A Differentiable Physics Engine for Large Scale Rigid Body Simulation}},
  author = {C. Daniel Freeman and Erik Frey and Anton Raichuk and Sertan Girgin and Igor Mordatch and Olivier Bachem},
  year = {2021},
  eprint = {2106.13281},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2106.13281},
}
Paper论文 2021 · UC Berkeley · Google · FAIR NeurIPS 2021

Decision Transformer: Reinforcement Learning via Sequence Modeling

Recasts RL as sequence modeling — a GPT-style model predicts actions conditioned on the desired return; an early bridge between transformers and control.把强化学习改写成序列建模:GPT 式模型以期望回报为条件预测动作;是 Transformer 与控制结合的早期代表。

BibTeX
@misc{chen2021decision,
  title = {{Decision Transformer: Reinforcement Learning via Sequence Modeling}},
  author = {Lili Chen and Kevin Lu and Aravind Rajeswaran and Kimin Lee and Aditya Grover and Michael Laskin and Pieter Abbeel and Aravind Srinivas and Igor Mordatch},
  year = {2021},
  eprint = {2106.01345},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/2106.01345},
}
Paper论文 2021 · Google CoRL 2021

Implicit Behavioral Cloning

Represents the policy as an energy-based model over actions instead of a regressor, which handles multimodal and discontinuous demonstrations much better.用基于能量的模型(而非直接回归)表示策略,更好地处理多模态、不连续的示范数据。

BibTeX
@misc{florence2021implicit,
  title = {{Implicit Behavioral Cloning}},
  author = {Pete Florence and Corey Lynch and Andy Zeng and Oscar Ramirez and Ayzaan Wahid and Laura Downs and Adrian Wong and Johnny Lee and Igor Mordatch and Jonathan Tompson},
  year = {2021},
  eprint = {2109.00137},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2109.00137},
}
Paper论文 2021 · Google Research, DeepMind ICLR 2021

Mastering Atari with Discrete World Models

DreamerV2 — switches Dreamer to categorical latent variables and learns Atari behaviour purely inside the world model, reaching human-level scores.DreamerV2:把 Dreamer 的潜变量改为离散类别分布,仅在世界模型的想象中学习 Atari 策略,达到人类水平。

BibTeX
@misc{hafner2020mastering,
  title = {{Mastering Atari with Discrete World Models}},
  author = {Danijar Hafner and Timothy Lillicrap and Mohammad Norouzi and Jimmy Ba},
  year = {2020},
  eprint = {2010.02193},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/2010.02193},
}
World Models世界模型simmodel-based-rldiscrete-latents
Paper论文 2020 · Google CoRL 2020

Transporter Networks: Rearranging the Visual World for Robotic Manipulation

Transporter Nets — pick-and-place as spatial matching of deep features cropped around the pick point; very sample-efficient on rearrangement tasks, released with the Ravens benchmark.Transporter 网络:把抓取-放置建模为以抓取点为中心的深度特征的空间匹配,在重排类任务上样本效率很高,并随论文发布 Ravens 基准。

BibTeX
@misc{zeng2020transporter,
  title = {{Transporter Networks: Rearranging the Visual World for Robotic Manipulation}},
  author = {Andy Zeng and Pete Florence and Jonathan Tompson and Stefan Welker and Jonathan Chien and Maria Attarian and Travis Armstrong and Ivan Krasin and Dan Duong and Ayzaan Wahid and Vikas Sindhwani and Johnny Lee},
  year = {2020},
  eprint = {2010.14406},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2010.14406},
}
Manipulation & Dexterous Hands操作与灵巧手simrealpick-and-placerearrangementequivariance
Paper论文 2020 · DeepMind

dm_control: Software and Tasks for Continuous Control

DeepMind's Python stack on top of MuJoCo — bindings, PyMJCF and Composer for building models and tasks, the standard Control Suite benchmarks, plus locomotion and arm-and-bricks manipulation tasks.DeepMind 基于 MuJoCo 的 Python 工具栈:底层绑定、用于程序化建模与编写任务的 PyMJCF 和 Composer、作为标准基准的 Control Suite,以及运动控制和机械臂拼积木操作任务。

BibTeX
@misc{tassa2020dm,
  title = {{dm_control: Software and Tasks for Continuous Control}},
  author = {Yuval Tassa and Saran Tunyasuvunakool and Alistair Muldal and Yotam Doron and Piotr Trochim and Siqi Liu and Steven Bohez and Josh Merel and Tom Erez and Timothy Lillicrap and Nicolas Heess},
  year = {2020},
  eprint = {2006.12983},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2006.12983},
}
Paper论文 2019 · University of Toronto, Google

Dream to Control: Learning Behaviors by Latent Imagination

The first Dreamer — learns a latent world model from pixels and trains the policy entirely on imagined rollouts.第一代 Dreamer:从像素学习潜空间世界模型,并完全在想象的轨迹上训练策略。

BibTeX
@misc{hafner2019dream,
  title = {{Dream to Control: Learning Behaviors by Latent Imagination}},
  author = {Danijar Hafner and Timothy Lillicrap and Jimmy Ba and Mohammad Norouzi},
  year = {2019},
  eprint = {1912.01603},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/1912.01603},
}
World Models世界模型model-based-rllatent-dynamics
Paper论文 2019 · Google Brain, DeepMind ICML 2019

Learning Latent Dynamics for Planning from Pixels

PlaNet — learns a latent dynamics model (the RSSM later reused by Dreamer) from images and plans actions directly in latent space with CEM.PlaNet:从图像中学习潜空间动力学模型(即后来 Dreamer 沿用的 RSSM),并用 CEM 直接在潜空间中规划动作。

BibTeX
@misc{hafner2018learning,
  title = {{Learning Latent Dynamics for Planning from Pixels}},
  author = {Danijar Hafner and Timothy Lillicrap and Ian Fischer and Ruben Villegas and David Ha and Honglak Lee and James Davidson},
  year = {2018},
  eprint = {1811.04551},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/1811.04551},
}
World Models世界模型simmodel-based-rlplanninglatent-dynamics
Paper论文 2018 · Google CoRL 2018

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation

Off-policy Q-learning with a cross-entropy-method action optimizer, trained on several hundred thousand real grasp attempts collected by a fleet of robots; reports 96% grasp success on unseen objects.使用交叉熵方法优化动作的离策略 Q 学习,在机器人集群采集的数十万次真实抓取上训练;报告对未见物体的抓取成功率为 96%。

BibTeX
@misc{kalashnikov2018qt,
  title = {{QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation}},
  author = {Dmitry Kalashnikov and Alex Irpan and Peter Pastor and Julian Ibarz and Alexander Herzog and Eric Jang and Deirdre Quillen and Ethan Holly and Mrinal Kalakrishnan and Vincent Vanhoucke and Sergey Levine},
  year = {2018},
  eprint = {1806.10293},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/1806.10293},
}
Paper论文 2018 · Google Brain, IDSIA

World Models

The classic VAE + RNN agent that learns a compressed model of its environment and trains a controller inside the "dream".经典的 VAE + RNN 智能体,学习环境的压缩模型并在"梦境"中训练控制器。

BibTeX
@misc{ha2018world,
  title = {{World Models}},
  author = {David Ha and Jürgen Schmidhuber},
  year = {2018},
  eprint = {1803.10122},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/1803.10122},
}
World Models世界模型simmodel-based-rlvaernn