📡 Embodied AI Radar具身智能雷达

Labs机构 › UC San Diego加州大学圣地亚哥分校

UC San Diego · embodied AI & robot learning加州大学圣地亚哥分校 · 具身智能与机器人学习

12 curated papers, code, datasets and hardware from UC San Diego (7 with code), with one-line takeaways.12 个来自加州大学圣地亚哥分校的精选论文、代码、数据集与硬件(7 个附代码),均配一句话要点。

Labs are matched from each entry's affiliation field by simple name rules (data/labs.yaml); joint work appears under every lab named. Only curated entries are listed, because daily arXiv papers carry no affiliations.机构由每个条目的机构字段按名称规则匹配(data/labs.yaml),合作工作会出现在所有署名机构下。仅列出精选条目,因为每日 arXiv 论文不含机构信息。

By topic:按主题: VLA & Robot Foundation ModelsVLA 与机器人基础模型 (2) · Manipulation & Dexterous Hands操作与灵巧手 (4) · Tactile & Visuotactile Sensing触觉与视触觉感知 (2) · Locomotion & Humanoids运动控制与人形机器人 (3) · World Models世界模型 (2) · Navigation & Embodied Agents导航与具身智能体 (1) · Planning, Control & Safety规划、控制与安全 (1) · Sim-to-Real & Simulators仿真与 Sim-to-Real (2) · Datasets & Benchmarks数据集与基准 (1) · Hardware & Open-Source Robots硬件与开源机器人 (3)

All entries全部条目 (12)

Hardware硬件 2024 · UC San Diego, HKU

Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning

Apple Vision Pro teleoperation of two dexterous robot hands with real-time retargeting, collision and singularity avoidance, and low-cost vibration haptics for the operator.用 Apple Vision Pro 实时遥操作双灵巧手,带实时重定向、碰撞与奇异点规避,并为操作者提供低成本振动触觉反馈。

BibTeX
@misc{ding2024bunny,
  title = {{Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning}},
  author = {Runyu Ding and Yuzhe Qin and Jiyue Zhu and Chengzhe Jia and Shiqi Yang and Ruihan Yang and Xiaojuan Qi and Xiaolong Wang},
  year = {2024},
  eprint = {2407.03162},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2407.03162},
}
Hardware & Open-Source Robots硬件与开源机器人Manipulation & Dexterous Hands操作与灵巧手Robot arm机械臂Dexterous hand灵巧手teleoperationvrhandbimanualhaptics
Benchmark基准 2024 · UC San Diego, Stanford, UC Berkeley, Google DeepMind

Evaluating Real-World Robot Manipulation Policies in Simulation

SimplerEnv — real-to-sim environments whose scores correlate with real-robot results for RT-1/Octo-style policies.SimplerEnv:真实到仿真的评测环境,其得分与 RT-1/Octo 类策略的真机表现相关。

BibTeX
@misc{li2024evaluating,
  title = {{Evaluating Real-World Robot Manipulation Policies in Simulation}},
  author = {Xuanlin Li and Kyle Hsu and Jiayuan Gu and Karl Pertsch and Oier Mees and Homer Rich Walke and Chuyuan Fu and Ishikaa Lunawat and Isabel Sieh and Sean Kirmani and Sergey Levine and Jiajun Wu and Chelsea Finn and Hao Su and Quan Vuong and Ted Xiao},
  year = {2024},
  eprint = {2405.05941},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2405.05941},
}
Paper论文 2024 · UC San Diego

Expressive Whole-Body Control for Humanoid Robots

Imitates upper-body human motion capture while keeping robust legs, giving humanoids expressive gestures and dance.上半身模仿人类动捕、下肢保持鲁棒行走,让人形机器人做出富有表现力的动作和舞蹈。

BibTeX
@misc{cheng2024expressive,
  title = {{Expressive Whole-Body Control for Humanoid Robots}},
  author = {Xuxin Cheng and Yandong Ji and Junming Chen and Ruihan Yang and Ge Yang and Xiaolong Wang},
  year = {2024},
  eprint = {2402.16796},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2402.16796},
}
Locomotion & Humanoids运动控制与人形机器人Humanoid人形sim2realhumanoidrlmotion-imitation
Paper论文 2024 · UC San Diego

ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI

GPU-parallel simulation and rendering with many manipulation tasks; fast visual RL and data generation.GPU 并行仿真与渲染,内置大量操作任务,适合快速视觉强化学习与数据生成。

BibTeX
@misc{tao2024maniskill,
  title = {{ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI}},
  author = {Stone Tao and Fanbo Xiang and Arth Shukla and Yuzhe Qin and Xander Hinrichsen and Xiaodi Yuan and Chen Bao and Xinsong Lin and Yulin Liu and Tse-kai Chan and Yuan Gao and Xuanlin Li and Tongzhou Mu and Nan Xiao and Arnav Gurha and Viswesh Nagaswamy Rajesh and Yong Woo Choi and Yen-Ru Chen and Zhiao Huang and Roberto Calandra and Rui Chen and Shan Luo and Hao Su},
  year = {2024},
  eprint = {2410.00425},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2410.00425},
}
Sim-to-Real & Simulators仿真与 Sim-to-Realgpu-parallelrenderingbenchmarkopen-source
Paper论文 2024 · UC San Diego, USC, NVIDIA RSS 2025

NaVILA: Legged Robot Vision-Language-Action Model for Navigation

A VLA outputs mid-level navigation commands in language that an RL locomotion policy executes, so legged robots follow natural-language routes; adds Isaac Lab VLN benchmarks.VLA 先用语言输出中层导航指令,再由强化学习运动策略执行,让腿式机器人按自然语言指令导航;并提供 Isaac Lab 上的 VLN 基准。

BibTeX
@misc{cheng2024navila,
  title = {{NaVILA: Legged Robot Vision-Language-Action Model for Navigation}},
  author = {An-Chieh Cheng and Yandong Ji and Zhaojing Yang and Zaitian Gongye and Xueyan Zou and Jan Kautz and Erdem Bıyık and Hongxu Yin and Sifei Liu and Xiaolong Wang},
  year = {2024},
  eprint = {2412.04453},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2412.04453},
}
Hardware硬件 2024 · UC San Diego, MIT

Open-TeleVision: Teleoperation with Immersive Active Visual Feedback

VR teleoperation with an actively steered stereo camera, used to collect humanoid manipulation data.带主动转向立体相机的 VR 遥操作系统,用于采集人形机器人操作数据。

BibTeX
@misc{cheng2024open,
  title = {{Open-TeleVision: Teleoperation with Immersive Active Visual Feedback}},
  author = {Xuxin Cheng and Jialong Li and Shiqi Yang and Ge Yang and Xiaolong Wang},
  year = {2024},
  eprint = {2407.01512},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2407.01512},
}
Hardware硬件 2023 · UC San Diego, NVIDIA RSS 2023

AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System

Camera-only teleoperation for arm + dexterous hand pairs - tracks the operator's hand and retargets it to many different robot hands, in sim or on real hardware.仅用相机即可遥操作"机械臂 + 灵巧手":跟踪操作者手部并重定向到多种不同的机器人手,可用于仿真或真机。

BibTeX
@misc{qin2023anyteleop,
  title = {{AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System}},
  author = {Yuzhe Qin and Wei Yang and Binghao Huang and Karl Van Wyk and Hao Su and Xiaolong Wang and Yu-Wei Chao and Dieter Fox},
  year = {2023},
  eprint = {2307.04577},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2307.04577},
}
Hardware & Open-Source Robots硬件与开源机器人Manipulation & Dexterous Hands操作与灵巧手Robot arm机械臂Dexterous hand灵巧手teleoperationhandvision-basedretargeting
Paper论文 2023 · UC San Diego

Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing

Turns tactile readings into points in the same 3D frame as the camera point cloud and the hand model, so one visuotactile point-cloud input drives in-hand object rotation.将触觉读数转换为与相机点云、手部模型处于同一三维坐标系的点,用统一的视触觉点云输入驱动手内物体旋转。

BibTeX
@misc{yuan2023robot,
  title = {{Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing}},
  author = {Ying Yuan and Haichuan Che and Yuzhe Qin and Binghao Huang and Zhao-Heng Yin and Kang-Won Lee and Yi Wu and Soo-Chul Lim and Xiaolong Wang},
  year = {2023},
  eprint = {2312.01853},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2312.01853},
}
Tactile & Visuotactile Sensing触觉与视触觉感知Manipulation & Dexterous Hands操作与灵巧手Dexterous hand灵巧手dexterous-handvisuotactilepoint-cloudsim2realrl
Paper论文 2023 · UC San Diego RSS 2023

Rotating without Seeing: Towards In-hand Dexterity through Touch

Covers a robot hand with cheap binary (contact / no contact) touch sensors and trains an RL policy in simulation that rotates unseen objects in hand on a real robot without any vision.在灵巧手上铺设廉价的二值(有/无接触)触觉传感器,在仿真中用强化学习训练策略,迁移到真机后无需视觉即可在手中旋转未见过的物体。

BibTeX
@misc{yin2023rotating,
  title = {{Rotating without Seeing: Towards In-hand Dexterity through Touch}},
  author = {Zhao-Heng Yin and Binghao Huang and Yuzhe Qin and Qifeng Chen and Xiaolong Wang},
  year = {2023},
  eprint = {2303.10880},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2303.10880},
}
Paper论文 2023 · UC San Diego

TD-MPC2: Scalable, Robust World Models for Continuous Control

An implicit (decoder-free) world model planned with MPC; one set of hyperparameters works across many continuous-control tasks and scales to a multitask agent.无解码器的隐式世界模型配合 MPC 规划;同一组超参数适用于大量连续控制任务,并可扩展为多任务智能体。

BibTeX
@misc{hansen2023td,
  title = {{TD-MPC2: Scalable, Robust World Models for Continuous Control}},
  author = {Nicklas Hansen and Hao Su and Xiaolong Wang},
  year = {2023},
  eprint = {2310.16828},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/2310.16828},
}
Paper论文 2022 · UC San Diego ICML 2022

Temporal Difference Learning for Model Predictive Control

TD-MPC — a task-oriented latent model used for short-horizon planning, with a learned value function covering the long-term return; the predecessor of TD-MPC2.TD-MPC:用面向任务的潜空间模型做短视距规划,由学习到的价值函数估计长期回报;是 TD-MPC2 的前身。

BibTeX
@misc{hansen2022temporal,
  title = {{Temporal Difference Learning for Model Predictive Control}},
  author = {Nicklas Hansen and Xiaolong Wang and Hao Su},
  year = {2022},
  eprint = {2203.04955},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url = {https://arxiv.org/abs/2203.04955},
}
World Models世界模型simmodel-based-rlmpcplanning
Paper论文 2020 · UC San Diego

SAPIEN: A SimulAted Part-based Interactive ENvironment

A simulator with large-scale articulated objects (PartNet-Mobility) for interaction tasks like opening drawers.包含大规模铰接物体(PartNet-Mobility)的仿真器,适合开抽屉等交互任务。

BibTeX
@misc{xiang2020sapien,
  title = {{SAPIEN: A SimulAted Part-based Interactive ENvironment}},
  author = {Fanbo Xiang and Yuzhe Qin and Kaichun Mo and Yikuan Xia and Hao Zhu and Fangchen Liu and Minghua Liu and Hanxiao Jiang and Yifu Yuan and He Wang and Li Yi and Angel X. Chang and Leonidas J. Guibas and Hao Su},
  year = {2020},
  eprint = {2003.08515},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url = {https://arxiv.org/abs/2003.08515},
}