ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation

ReKep — a VLM writes manipulation tasks as Python cost functions over 3D keypoints (relational keypoint constraints), solved by a hierarchical optimizer in real time; zero-shot, no task-specific training.ReKep：由视觉语言模型把操作任务写成定义在三维关键点上的 Python 代价函数（关系关键点约束），再由分层优化器实时求解；零样本，无需针对任务训练。

Paper论文 · Code代码 ★ 992

BibTeX

@misc{huang2024rekep,
  title = {{ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation}},
  author = {Wenlong Huang and Chen Wang and Yunzhu Li and Ruohan Zhang and Li Fei-Fei},
  year = {2024},
  eprint = {2409.01652},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2409.01652},
}

Manipulation & Dexterous Hands操作与灵巧手 VLA & Robot Foundation ModelsVLA 与机器人基础模型realzero-shotkeypointsvlmoptimization

Authors (from arXiv): 作者（来自 arXiv）：Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, Li Fei-Fei

More from: 同机构更多条目：Stanford斯坦福大学 · Columbia University哥伦比亚大学

ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation

More in Manipulation & Dexterous Hands更多操作与灵巧手条目