Peking University · embodied AI & robot learning北京大学 · 具身智能与机器人学习
11 curated papers, code, datasets and hardware from Peking University (10 with code), with one-line takeaways.11 个来自北京大学的精选论文、代码、数据集与硬件(10 个附代码),均配一句话要点。
Labs are matched from each entry's affiliation field by simple name rules (data/labs.yaml); joint work appears under every lab named. Only curated entries are listed, because daily arXiv papers carry no affiliations.机构由每个条目的机构字段按名称规则匹配(data/labs.yaml),合作工作会出现在所有署名机构下。仅列出精选条目,因为每日 arXiv 论文不含机构信息。
Isaac Sim benchmark with 5 arms, 6 dexterous hands, ~200 assets and 18k human demos over 66 tasks; reports the best trained policy (a WAM) at only 29.1% on Core-50, near 0% on precision-fit and deformable tasks.基于 Isaac Sim 的灵巧操作基准:5 种机械臂、6 种灵巧手、约 200 个资产、1.8 万条人类演示、66 个任务;报告最好的训练策略(一种世界动作模型)在 Core-50 上仅 29.1%,精密装配和柔性物体任务接近 0%。
@misc{luo2026handson,
title = {{HandsOn: A Large-Scale Simulation Benchmark for Generalizable Dexterous Manipulation}},
author = {Tianwei Luo and Yiming He and Hongjin Wu and Ting Pan and Debin Ren and Haorui Guo and Junlin Yang and Yuxin Zuo and Zhehao Kou and Kaisen Yang and Zhuoqun Lai and Yi Ding and Jiale Yu and Yuchen Zhang and Zijie Qiu and Haozhan Li and Ning Ding},
year = {2026},
eprint = {2610.26765},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.26765},
}
A pretrained VLM plans and a diffusion controller grasps with a dexterous hand, working on domain-invariant features from foundation models; reports 90%+ grasp success across thousands of unseen cluttered scenes.由预训练 VLM 做高层规划、扩散控制器驱动灵巧手抓取,底层输入为基础模型提取的域不变特征;报告在数千个未见过的杂乱场景中抓取成功率超过 90%。
@misc{zhong2025dexgraspvla,
title = {{DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping}},
author = {Yifan Zhong and Xuchuan Huang and Ruochong Li and Ceyao Zhang and Zhang Chen and Tianrui Guan and Fanlian Zeng and Ka Num Lui and Yuyao Ye and Yitao Liang and Yaodong Yang and Yuanpei Chen},
year = {2025},
eprint = {2502.20900},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2502.20900},
}
Pretrains a grasping VLA almost only on SynGrasp-1B, a billion-frame photorealistic sim dataset, co-trained with web data; reports zero-shot sim-to-real open-vocabulary grasping and few-shot adaptation.几乎只用十亿帧级的照片级仿真抓取数据集 SynGrasp-1B(并联合网络语义数据)预训练抓取 VLA;论文报告可零样本从仿真迁移到真实开放词汇抓取,并支持少样本适配。
@misc{deng2025graspvla,
title = {{GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data}},
author = {Shengliang Deng and Mi Yan and Songlin Wei and Haixin Ma and Yuxin Yang and Jiayi Chen and Zhiqi Zhang and Taoyu Yang and Xuheng Zhang and Wenhao Zhang and Heming Cui and Zhizheng Zhang and He Wang},
year = {2025},
eprint = {2505.03233},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2505.03233},
}
Open-source 'brain–cerebellum' stack: a RoboBrain MLLM plans and assigns tasks across heterogeneous robots, per-robot skill libraries execute, and a shared spatial/temporal/robot memory keeps them in sync.开源的“大脑–小脑”分层系统:RoboBrain 多模态大模型负责跨异构机器人的任务规划与分配,各机器人上的技能库负责执行,共享的空间/时间/本体记忆保持多机协同。
@misc{tan2025roboos,
title = {{RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration}},
author = {Huajie Tan and Xiaoshuai Hao and Cheng Chi and Minglan Lin and Yaoxu Lyu and Mingyu Cao and Dong Liang and Zhuo Chen and Mengsi Lyu and Cheng Peng and Chenrui He and Yulong Ao and Yonghua Lin and Pengwei Wang and Zhongyuan Wang and Shanghang Zhang},
year = {2025},
eprint = {2505.03673},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2505.03673},
}
MetaSim puts many simulators (Isaac Sim/Gym, MuJoCo, Genesis, SAPIEN…) behind one interface; adds a large synthetic dataset migrated from public benchmarks and unified IL/RL benchmarks.MetaSim 用统一接口封装多种仿真器(Isaac Sim/Gym、MuJoCo、Genesis、SAPIEN、PyBullet 等);提供由公开基准迁移而来的大规模合成数据集,以及按泛化层级划分的统一模仿学习/强化学习基准。
@misc{geng2025roboverse,
title = {{RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning}},
author = {Haoran Geng and Feishi Wang and Songlin Wei and Yuyang Li and Bangjun Wang and Boshi An and Charlie Tianyue Cheng and Haozhe Lou and Peihao Li and Yen-Jen Wang and Yutong Liang and Dylan Goetting and Chaoyi Xu and Haozhe Chen and Yuxi Qian and Yiran Geng and Jiageng Mao and Weikang Wan and Mingtong Zhang and Jiangran Lyu and Siheng Zhao and Jiazhao Zhang and Jialiang Zhang and Chengyang Zhao and Haoran Lu and Yufei Ding and Ran Gong and Yuran Wang and Yuxuan Kuang and Ruihai Wu and Baoxiong Jia and Carlo Sferrazza and Hao Dong and Siyuan Huang and Yue Wang and Jitendra Malik and Pieter Abbeel},
year = {2025},
eprint = {2504.18904},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2504.18904},
}
Online RL for VLAs with GRPO and only binary success rewards on OpenVLA-OFT, plus exploration tweaks; reports 99.1% on LIBERO and 70.4% on RoboTwin 1.0 (vs 39.8% for the SFT baseline).在 OpenVLA-OFT 上用 GRPO 与仅"成功/失败"二值奖励做在线强化学习,并加入若干促进探索的改动;论文报告 LIBERO 成功率 99.1%、RoboTwin 1.0 为 70.4%(SFT 基线 39.8%)。
@misc{li2025simplevla,
title = {{SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning}},
author = {Haozhan Li and Yuxin Zuo and Jiale Yu and Yuhao Zhang and Zhaohui Yang and Kaiyan Zhang and Xuekai Zhu and Yuchen Zhang and Tianxing Chen and Ganqu Cui and Dehui Wang and Dingxiang Luo and Yuchen Fan and Youbang Sun and Jia Zeng and Jiangmiao Pang and Shanghang Zhang and Yu Wang and Yao Mu and Bowen Zhou and Ning Ding},
year = {2025},
eprint = {2509.09674},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2509.09674},
}
A video VLM that reads the monocular RGB history plus the instruction and outputs the next navigation action directly, without maps, odometry or depth.基于视频的视觉语言模型:只看单目 RGB 历史帧和语言指令,直接输出下一步导航动作,不依赖地图、里程计或深度。
@misc{zhang2024navid,
title = {{NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation}},
author = {Jiazhao Zhang and Kunyu Wang and Rongtao Xu and Gengze Zhou and Yicong Hong and Xiaomeng Fang and Qi Wu and Zhizheng Zhang and He Wang},
year = {2024},
eprint = {2402.15852},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2402.15852},
}
Seer forecasts the next visual state and feeds it to an inverse dynamics model that outputs actions, trained end to end and pretrained on DROID; reports 4.28 avg. length on CALVIN ABC-D plus LIBERO and real-robot gains.Seer 先预测未来视觉状态,再由逆动力学模型据此输出动作,端到端训练并在 DROID 上预训练;报告 CALVIN ABC-D 平均任务长度 4.28,在 LIBERO-LONG 和真机任务上也有提升。
@misc{tian2024predictive,
title = {{Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation}},
author = {Yang Tian and Sizhe Yang and Jia Zeng and Ping Wang and Dahua Lin and Hao Dong and Jiangmiao Pang},
year = {2024},
eprint = {2412.15109},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2412.15109},
}
Large real-robot teleoperation dataset across several embodiments (single-arm, dual-arm, humanoid) collected under one standardized protocol, and it keeps failed demos with annotated causes.在统一规范下采集的大规模真机遥操作数据集,覆盖单臂、双臂与人形等多种本体,并保留了标注失败原因的失败演示。
@misc{wu2024robomind,
title = {{RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation}},
author = {Kun Wu and Chengkai Hou and Jiaming Liu and Zhengping Che and Xiaozhu Ju and Zhuqin Yang and Meng Li and Yinuo Zhao and Zhiyuan Xu and Guang Yang and Shichao Fan and Xinhua Wang and Fei Liao and Zhen Zhao and Guangyu Li and Zhao Jin and Lecheng Wang and Jilei Mao and Ning Liu and Pei Ren and Qiang Zhang and Yaoxu Lyu and Mengzhen Liu and Jingyang He and Yulin Luo and Zeyu Gao and Chenxuan Li and Chenyang Gu and Yankai Fu and Di Wu and Xingyu Wang and Sixiang Chen and Zhenyu Wang and Pengju An and Siyuan Qian and Shanghang Zhang and Jian Tang},
year = {2024},
eprint = {2412.13877},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2412.13877},
}
Extends NaVid into one video VLA model for several navigation tasks (instruction following, object search, question answering, human following).把 NaVid 扩展为单一视频 VLA 模型,统一处理多种导航任务(指令导航、目标搜索、具身问答、跟随人)。
@misc{zhang2024uni,
title = {{Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks}},
author = {Jiazhao Zhang and Kunyu Wang and Shaoan Wang and Minghan Li and Haoran Liu and Songlin Wei and Zhongyuan Wang and Zhizheng Zhang and He Wang},
year = {2024},
eprint = {2412.06224},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2412.06224},
}
Over a million physically validated ShadowHand grasps on thousands of objects, synthesized by optimizing a differentiable force-closure energy; a standard training set for dexterous grasp models.通过优化可微的力封闭能量合成、并经物理仿真验证的百万级 ShadowHand 抓取,覆盖数千个物体,是灵巧抓取模型的常用训练集。
@misc{wang2022dexgraspnet,
title = {{DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation}},
author = {Ruicheng Wang and Jialiang Zhang and Jiayi Chen and Yinzhen Xu and Puhao Li and Tengyu Liu and He Wang},
year = {2022},
eprint = {2210.02697},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2210.02697},
}