The map of embodied AI & robot learning具身智能与机器人学习全景图
Hand-verified papers, code, datasets, simulators and hardware with one-line takeaways in English and Chinese — plus every new robotics paper on arXiv, tagged by topic, every day.人工核验的论文、代码、数据集、仿真器与硬件，配中英文一句话要点；外加每天自动收录并按主题标注的 arXiv 机器人新论文。
S² is a training-free offline metric of how well a model's representation aligns with physical motion. Reports it tracks robot task success across model families and can rank VLM backbones before VLA fine-tuning.S² 是一种无需训练的离线指标，衡量模型视觉表征与物理运动几何的对齐程度（把柏拉图表征假说扩展到机器人）；报告它与多类模型的真实任务成功率正相关，可在微调成 VLA 之前筛选 VLM 骨干。
@misc{xie2026sensorimotor,
title = {{Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success}},
author = {Yunfei Xie and Howard H. Qian and Shiyi Lan and Kaiyu Hang and Chen Wei},
year = {2026},
eprint = {2610.31670},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.31670},
}
PAL has a high-level VLM write future-relevant subtasks into an executable memory while watching, then hand the current one to a low-level VLA. Reports 56.05% vs 44.51% on RoboMME and 60.83% on 4 real tasks.PAL 让高层 VLM 在观察过程中把未来需要的子任务提前写入可执行记忆（subtask_list），执行时再挑出当前子任务交给底层 VLA；报告在 RoboMME 上成功率 56.05%（最强基线 44.51%），4 个真实任务总体 60.83%。
@misc{liu2026plan,
title = {{Plan First, Act Later: Streaming Reasoning for Executable Memory in Robot Control}},
author = {Qinghang Liu and Conglin Wang and Zongsheng Li and Boyong He and Han Gao and Pengcheng Wang and Tao Sun and Lige Liu and Chang Liu},
year = {2026},
eprint = {2610.29329},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.29329},
}
SpatialDex pre-trains a VLM-based dexterous policy on scene geometry and hand motion reconstructed from 1M RGB-only human clips. Reports +38.3 pts over prior methods on 6 real bimanual tasks.SpatialDex 从 100 万段纯 RGB 人类视频中重建场景几何与手部运动，并以几何作为输入和预测目标预训练基于 VLM 的灵巧策略；报告在 9 个仿真任务上比现有方法高 16.5 分、6 个真实双臂灵巧任务上高 38.3 分。
@misc{he2026reconstruction,
title = {{Reconstruction-Augmented Visuomotor Pre-Training of 3D-Grounded Bimanual Dexterity from RGB-Only Human Videos}},
author = {Xingyi He and Zhiwen Qiu and Adhitya Polavaram and Yunhao Cao and Cory Fan and Julius Samwer and Saleh Kalantari and Kuan Fang},
year = {2026},
eprint = {2610.29716},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.29716},
}
SMR distils physics experts that track robot-aware targets into a strictly causal Transformer, retargeting human motion to humanoids frame by frame. Reports better closed-loop results than kinematic baselines.SMR 先让物理专家在仿真中跟踪由人体动作转换来的机器人任务空间目标，再用其执行轨迹训练严格因果的 Transformer，实现逐帧流式的人形动作重定向；报告在相同全身控制器下闭环表现优于运动学与物理精修基线。
@misc{gu2026smr,
title = {{SMR: Learning streaming motion retargeting for humanoids from human semantic}},
author = {Chun Gu and Juntong Zhang and Jingyu Li and Li Zhang},
year = {2026},
eprint = {2610.29169},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.29169},
}
GEMS keeps a 3D Gaussian Splatting map that is updated only where the scene changes, and an agent renders views on demand to answer when/where questions. Introduces GEMS-Bench (15 scenes, 473 QA pairs).GEMS 维护一张只在场景变化处增量更新的 3DGS 地图，智能体按需渲染新视角来回答时空问题和定位物体；同时发布 GEMS-Bench（3 个真实 + 12 个合成环境，473 个问答对），报告在物体定位和时空问答上优于近期基线。
@misc{chen2026gaussian,
title = {{Gaussian Episodic Memory for Long-Term Spatiotemporal Reasoning}},
author = {Ling-Chia Chen and Abdelrhman Werby and Kaan Aytekin and Kai O. Arras},
year = {2026},
eprint = {2610.33158},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.33158},
}
FLEX lets language corrections edit a robot's task features via add, generalize and mask (not add-only). Reports higher normalized reward than other editing methods in 2D nav, Franka tabletop and a real user study.FLEX 用自然语言纠正来迭代修改机器人的任务特征集合，支持添加、泛化、屏蔽三种操作（而非只能添加）；在二维导航、Franka 桌面操作和真实用户研究中报告比其他特征编辑方法获得更高的归一化奖励。
@misc{lee2026flexible,
title = {{Flexible Language-Guided Representation Editing for Adaptive Robot Behavior}},
author = {Audrey Lee and Nathaniel Dennler and Ilia Sucholutsky and Andreea Bobu},
year = {2026},
eprint = {2610.28893},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.28893},
}
Robotics papers from our feed that made Hugging Face Daily Papers in the last 14 days, by community upvotes.最近 14 天登上 Hugging Face Daily Papers 的本站收录论文，按社区点赞数排序。
@misc{brohan2023rt,
title = {{RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control}},
author = {Anthony Brohan and Noah Brown and Justice Carbajal and Yevgen Chebotar and Xi Chen and Krzysztof Choromanski and Tianli Ding and Danny Driess and Avinava Dubey and Chelsea Finn and Pete Florence and Chuyuan Fu and Montse Gonzalez Arenas and Keerthana Gopalakrishnan and Kehang Han and Karol Hausman and Alexander Herzog and Jasmine Hsu and Brian Ichter and Alex Irpan and Nikhil Joshi and Ryan Julian and Dmitry Kalashnikov and Yuheng Kuang and Isabel Leal and Lisa Lee and Tsang-Wei Edward Lee and Sergey Levine and Yao Lu and Henryk Michalewski and Igor Mordatch and Karl Pertsch and Kanishka Rao and Krista Reymann and Michael Ryoo and Grecia Salazar and Pannag Sanketi and Pierre Sermanet and Jaspiar Singh and Anikait Singh and Radu Soricut and Huong Tran and Vincent Vanhoucke and Quan Vuong and Ayzaan Wahid and Stefan Welker and Paul Wohlhart and Jialin Wu and Fei Xia and Ted Xiao and others},
year = {2023},
eprint = {2307.15818},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2307.15818},
}
A 7B open VLA built on Llama 2 + DINOv2/SigLIP; the de-facto open baseline for VLA research and LoRA fine-tuning.基于 Llama 2 与 DINOv2/SigLIP 的 7B 开源 VLA，是 VLA 研究与 LoRA 微调的事实开源基线。
@misc{kim2024openvla,
title = {{OpenVLA: An Open-Source Vision-Language-Action Model}},
author = {Moo Jin Kim and Karl Pertsch and Siddharth Karamcheti and Ted Xiao and Ashwin Balakrishna and Suraj Nair and Rafael Rafailov and Ethan Foster and Grace Lam and Pannag Sanketi and Quan Vuong and Thomas Kollar and Benjamin Burchfiel and Russ Tedrake and Dorsa Sadigh and Sergey Levine and Percy Liang and Chelsea Finn},
year = {2024},
eprint = {2406.09246},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2406.09246},
}
A VLM backbone with a flow-matching action expert, trained across many robots, doing long dexterous tasks like laundry folding.VLM 主干加流匹配动作专家，跨多种机器人训练，能完成叠衣服等长时程灵巧任务。
@misc{black2024vision,
title = {{$π_0$: A Vision-Language-Action Flow Model for General Robot Control}},
author = {Kevin Black and Noah Brown and Danny Driess and Adnan Esmail and Michael Equi and Chelsea Finn and Niccolo Fusai and Lachy Groom and Karol Hausman and Brian Ichter and Szymon Jakubczak and Tim Jones and Liyiming Ke and Sergey Levine and Adrian Li-Bell and Mohith Mothukuri and Suraj Nair and Karl Pertsch and Lucy Xiaoyang Shi and James Tanner and Quan Vuong and Anna Walling and Haohuan Wang and Ury Zhilinsky},
year = {2024},
eprint = {2410.24164},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2410.24164},
}
RECAP trains a VLA with RL via advantage conditioning on demos, on-robot rollouts and teleop corrections; reports it more than doubles throughput on hard tasks like laundry folding and espresso making.RECAP 通过优势条件化，用演示、机器人自主运行数据和遥操作纠正对 VLA 做强化学习；论文报告在叠衣服、做意式咖啡等难任务上吞吐量提升一倍以上、失败率约减半。
@misc{intelligence2025vla,
title = {{π*0.6: a VLA That Learns From Experience}},
author = {Physical Intelligence and Ali Amin and Raichelle Aniceto and Ashwin Balakrishna and Kevin Black and Ken Conley and Grace Connors and James Darpinian and Karan Dhabalia and Jared DiCarlo and Danny Driess and Michael Equi and Adnan Esmail and Yunhao Fang and Chelsea Finn and Catherine Glossop and Thomas Godden and Ivan Goryachev and Lachy Groom and Hunter Hancock and Karol Hausman and Gashon Hussein and Brian Ichter and Szymon Jakubczak and Rowan Jen and Tim Jones and Ben Katz and Liyiming Ke and Chandra Kuchi and Marinda Lamb and Devin LeBlanc and Sergey Levine and Adrian Li-Bell and Yao Lu and Vishnu Mano and Mohith Mothukuri and Suraj Nair and Karl Pertsch and Allen Z. Ren and Charvi Sharma and Lucy Xiaoyang Shi and Laura Smith and Jost Tobias Springenberg and Kyle Stachowicz and Will Stoeckle and Alex Swerdlow and James Tanner and Marcel Torne and Quan Vuong and Anna Walling and others},
year = {2025},
eprint = {2511.14759},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2511.14759},
}
Generating action sequences by denoising handles multimodal demos far better than regression; now a standard IL baseline.以去噪方式生成动作序列，比回归更能处理多模态演示，已成为模仿学习标准基线。
@misc{chi2023diffusion,
title = {{Diffusion Policy: Visuomotor Policy Learning via Action Diffusion}},
author = {Cheng Chi and Zhenjia Xu and Siyuan Feng and Eric Cousineau and Yilun Du and Benjamin Burchfiel and Russ Tedrake and Shuran Song},
year = {2023},
eprint = {2303.04137},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2303.04137},
}
@misc{zhao2023learning,
title = {{Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware}},
author = {Tony Z. Zhao and Vikash Kumar and Sergey Levine and Chelsea Finn},
year = {2023},
eprint = {2304.13705},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2304.13705},
}
@misc{chi2024universal,
title = {{Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots}},
author = {Cheng Chi and Zhenjia Xu and Chuer Pan and Eric Cousineau and Benjamin Burchfiel and Siyuan Feng and Russ Tedrake and Shuran Song},
year = {2024},
eprint = {2402.10329},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2402.10329},
}
@misc{lambeta2020digit,
title = {{DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation}},
author = {Mike Lambeta and Po-Wei Chou and Stephen Tian and Brian Yang and Benjamin Maloon and Victoria Rose Most and Dave Stroud and Raymond Santos and Ahmad Byagowi and Gregg Kammerer and Dinesh Jayaraman and Roberto Calandra},
year = {2020},
eprint = {2005.14679},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2005.14679},
}
Thousands of parallel GPU environments train a quadruped walking policy in minutes; legged_gym became a standard starter.数千个 GPU 并行环境几分钟就能训练出四足行走策略；legged_gym 成为标准入门框架。
@misc{rudin2021learning,
title = {{Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning}},
author = {Nikita Rudin and David Hoeller and Philipp Reist and Marco Hutter},
year = {2021},
eprint = {2109.11978},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2109.11978},
}
@misc{radosavovic2023real,
title = {{Real-World Humanoid Locomotion with Reinforcement Learning}},
author = {Ilija Radosavovic and Tete Xiao and Bike Zhang and Trevor Darrell and Jitendra Malik and Koushil Sreenath},
year = {2023},
eprint = {2303.03381},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2303.03381},
}
One motion-tracking recipe with shared hyperparameters learns agile skills (cartwheels, flip-kicks); a guided latent diffusion model composes them for new goals like joystick control, zero-shot on hardware.用同一套超参数的运动跟踪方案学会侧手翻、飞踢、冲刺等敏捷技能，再用带引导的潜空间扩散模型组合这些技能以完成摇杆遥控、避障等新目标，零样本迁移到真机。
@misc{liao2025beyondmimic,
title = {{BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion}},
author = {Qiayuan Liao and Takara E. Truong and Xiaoyu Huang and Yuman Gao and Guy Tevet and Koushil Sreenath and C. Karen Liu},
year = {2025},
eprint = {2508.08241},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2508.08241},
}
DreamerV3 — one set of hyperparameters works across 150+ tasks, including collecting diamonds in Minecraft from scratch.DreamerV3：一套超参数适用于 150 多个任务，包括从零开始在《我的世界》中挖到钻石。
@misc{hafner2023mastering,
title = {{Mastering Diverse Domains through World Models}},
author = {Danijar Hafner and Jurgis Pasukonis and Jimmy Ba and Timothy Lillicrap},
year = {2023},
eprint = {2301.04104},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2301.04104},
}
@misc{ahn2022do,
title = {{Do As I Can, Not As I Say: Grounding Language in Robotic Affordances}},
author = {Michael Ahn and Anthony Brohan and Noah Brown and Yevgen Chebotar and Omar Cortes and Byron David and Chelsea Finn and Chuyuan Fu and Keerthana Gopalakrishnan and Karol Hausman and Alex Herzog and Daniel Ho and Jasmine Hsu and Julian Ibarz and Brian Ichter and Alex Irpan and Eric Jang and Rosario Jauregui Ruano and Kyle Jeffrey and Sally Jesmonth and Nikhil J Joshi and Ryan Julian and Dmitry Kalashnikov and Yuheng Kuang and Kuang-Huei Lee and Sergey Levine and Yao Lu and Linda Luu and Carolina Parada and Peter Pastor and Jornell Quiambao and Kanishka Rao and Jarek Rettinghouse and Diego Reyes and Pierre Sermanet and Nicolas Sievers and Clayton Tan and Alexander Toshev and Vincent Vanhoucke and Fei Xia and Ted Xiao and Peng Xu and Sichun Xu and Mengyuan Yan and Andy Zeng},
year = {2022},
eprint = {2204.01691},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2204.01691},
}
The go-to feature-based SLAM library — monocular, stereo and RGB-D, with IMU fusion and multi-map relocalization.最常用的特征点 SLAM 库：支持单目、双目与 RGB-D，可融合 IMU，并支持多地图重定位。
@misc{campos2020orb,
title = {{ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM}},
author = {Carlos Campos and Richard Elvira and Juan J. Gómez Rodríguez and José M. M. Montiel and Juan D. Tardós},
year = {2020},
eprint = {2007.11898},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2007.11898},
}
SAM — a promptable segmentation model (points, boxes, masks) trained on 1B+ masks; a standard building block for object masks in robot perception pipelines.SAM：可用点、框、掩码提示的分割模型，在 10 亿级掩码上训练；已成为机器人感知流程中获取物体掩码的常用基础组件。
@misc{kirillov2023segment,
title = {{Segment Anything}},
author = {Alexander Kirillov and Eric Mintun and Nikhila Ravi and Hanzi Mao and Chloe Rolland and Laura Gustafson and Tete Xiao and Spencer Whitehead and Alexander C. Berg and Wan-Yen Lo and Piotr Dollár and Ross Girshick},
year = {2023},
eprint = {2304.02643},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2304.02643},
}
GPU-parallel motion generation for arms — many trajectory optimizations run at once, giving collision-free, smooth plans in milliseconds.面向机械臂的 GPU 并行运动生成：同时运行大量轨迹优化，在毫秒级给出无碰撞且平滑的轨迹。
@misc{sundaralingam2023curobo,
title = {{cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation}},
author = {Balakumar Sundaralingam and Siva Kumar Sastry Hari and Adam Fishman and Caelan Garrett and Karl Van Wyk and Valts Blukis and Alexander Millane and Helen Oleynikova and Ankur Handa and Fabio Ramos and Nathan Ratliff and Dieter Fox},
year = {2023},
eprint = {2310.17274},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2310.17274},
}
The clipped policy-gradient method behind most GPU-parallel sim-to-real locomotion and dexterity training.采用截断目标的策略梯度方法，是大多数 GPU 并行仿真训练（运动控制、灵巧操作）背后的算法。
HIL-SERL — real-world RL with human corrections, reported to reach near-perfect success on precise assembly and dynamic tasks after a few hours of training.HIL-SERL：结合人类纠正的真机强化学习，报告称在精密装配与动态任务上经数小时训练即可接近满分成功率。
@misc{luo2024precise,
title = {{Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning}},
author = {Jianlan Luo and Charles Xu and Jeffrey Wu and Sergey Levine},
year = {2024},
eprint = {2410.21845},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.21845},
}
1M+ trajectories from 22 robot types pooled by 30+ labs; RT-X models trained on it beat single-robot baselines.30 多个实验室汇集 22 种机器人的 100 多万条轨迹；在其上训练的 RT-X 优于单机器人基线。
@misc{collaboration2023open,
title = {{Open X-Embodiment: Robotic Learning Datasets and RT-X Models}},
author = {Open X-Embodiment Collaboration and Abby O'Neill and Abdul Rehman and Abhinav Gupta and Abhiram Maddukuri and Abhishek Gupta and Abhishek Padalkar and Abraham Lee and Acorn Pooley and Agrim Gupta and Ajay Mandlekar and Ajinkya Jain and Albert Tung and Alex Bewley and Alex Herzog and Alex Irpan and Alexander Khazatsky and Anant Rai and Anchit Gupta and Andrew Wang and Andrey Kolobov and Anikait Singh and Animesh Garg and Aniruddha Kembhavi and Annie Xie and Anthony Brohan and Antonin Raffin and Archit Sharma and Arefeh Yavary and Arhan Jain and Ashwin Balakrishna and Ayzaan Wahid and Ben Burgess-Limerick and Beomjoon Kim and Bernhard Schölkopf and Blake Wulfe and Brian Ichter and Cewu Lu and Charles Xu and Charlotte Le and Chelsea Finn and Chen Wang and Chenfeng Xu and Cheng Chi and Chenguang Huang and Christine Chan and Christopher Agia and Chuer Pan and Chuyuan Fu and Coline Devin and others},
year = {2023},
eprint = {2310.08864},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2310.08864},
}
@misc{liu2023libero,
title = {{LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning}},
author = {Bo Liu and Yifeng Zhu and Chongkai Gao and Yihao Feng and Qiang Liu and Yuke Zhu and Peter Stone},
year = {2023},
eprint = {2306.03310},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2306.03310},
}
PyTorch library with datasets, pretrained policies (ACT, Diffusion, SmolVLA) and drivers for low-cost arms.PyTorch 机器人学习库，提供数据集、预训练策略（ACT、Diffusion、SmolVLA）与低成本机械臂驱动。
Chaehyun Kim, Chi-Pin Huang, Jisoo Kim, Min-Hung Chen et al.
Abstract摘要
Vision-Language-Action (VLA) models have made substantial progress in robotic manipulation, but their generalization remains constrained by the diversity and coverage of action-labeled robot demonstrations. Human videos provide much broader behavioral and environmental experience, while learning from them faces a fundamental trade-off between scalable data and explicit action grounding. We introduce ThinkWorld, a VLA learning framework that grounds action reasoning in world dynamics and strengthens it through reinforcement learning with world imagination. ThinkWorld represents each action …
@misc{kim2026thinkworld,
title = {{ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning}},
author = {Chaehyun Kim and Chi-Pin Huang and Jisoo Kim and Min-Hung Chen and Yu-Chiang Frank Wang and Seungryong Kim and Fu-En Yang},
year = {2026},
eprint = {2610.37666},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37666},
}
Ian Chuang, Hiya Shah, Vamsi Chilamkur, Iman Soltani et al.
Abstract摘要
Dexterous manipulation with multi-fingered hands is the next frontier of robotics, yet learned hand skills remain limited to simple grasps or basic repeated motions, such as rotating an object. As robots attempt more demanding in-hand maneuvers, safety, such as not dropping the object, becomes critical, yet it is rarely measured. We study in-hand object pose reaching, a task that demands diverse and difficult grasps: moving an object to almost any position and orientation within reach of the fingers, at any wrist orientation, through long sequences of consecutive goals. We present a recipe …
@misc{chuang2026pushing,
title = {{Pushing Dexterity and Safety for In-Hand Object Pose Reaching That Works Remarkably Well}},
author = {Ian Chuang and Hiya Shah and Vamsi Chilamkur and Iman Soltani and Claire Tomlin},
year = {2026},
eprint = {2610.37652},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37652},
}
Tudor Braicu, Joseph DelPreto, Jared Boyer, M\'onika Farsang et al.
Abstract摘要
Agile robots require spatiotemporal intelligence to interpret scenes, reason about interactions, and anticipate future events. Yet it remains unclear how well foundation vision models support these capabilities, particularly from a moving egocentric camera, which is a natural view for robots. To investigate this, we collect an annotated tennis dataset and train a suite of vision models on spatiotemporal tasks. Our dataset contains egocentric video synchronized with five allocentric videos, estimated global positions, labeled contact times, and annotated stroke types. We then evaluate seven …
@misc{braicu2026perceiving,
title = {{Perceiving and Predicting Dynamic Sports: Spatiotemporal Intelligence in Vision Models for Egocentric Tennis Video}},
author = {Tudor Braicu and Joseph DelPreto and Jared Boyer and M\'onika Farsang and Maximilian St\"olzle and Tim Niklas Seyde and Daniela Rus},
year = {2026},
eprint = {2610.37619},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37619},
}
Ziyu Cao, William Talbot, Song Li, Pakpong Chirarattananon et al.
Abstract摘要
We present RESPLE2, a recursive multi-sensor state estimation framework using non-uniform B-splines for 6-DoF motion representation. By exploiting the knot-clamping boundary property, we propose a theoretically grounded mechanism for exact control-point prediction directly informed by inertial kinematics along the trajectory, with knot placement adapted to motion and measurement availability. Within an iterated extended Kalman filter, we perform this prediction using a subset of inertial measurements. A subsequent reset step preserves trajectory consistency as the knots extend. The update …
@misc{cao2026resple,
title = {{RESPLE2: Inertial-Informed Recursive Non-Uniform Spline Estimation for Multi-Sensor Odometry with Global Refinement}},
author = {Ziyu Cao and William Talbot and Song Li and Pakpong Chirarattananon and Fredrik Gustafsson and Marco Hutter and Kailai Li},
year = {2026},
eprint = {2610.37602},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37602},
}
Jianyu Hou, Jiawei Fu, Weikang Wan, Minghui Jiang et al.
Abstract摘要
Supernumerary robotic fingers augment human dexterity by enabling a single hand to perform complex "hold-and-manipulate" tasks that typically require bimanual coordination. However, existing devices are often hindered by bulky hardware that limits interaction to large objects and rely on explicit hand motion triggers that lack environmental context. To address these challenges, we present EgoThumb, a fully open-source, vision-driven supernumerary robotic thumb. EgoThumb features a compact, dual tendon-driven finger whose actuation is relocated to the forearm, reducing distal weight while …
@misc{hou2026egothumb,
title = {{EgoThumb: A Vision-Driven Supernumerary Robotic Thumb for Autonomous Grasp and Holding Assistance}},
author = {Jianyu Hou and Jiawei Fu and Weikang Wan and Minghui Jiang and Jiajun Li and Hao Su},
year = {2026},
eprint = {2610.37582},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37582},
}
Threading a quadrotor through a moving narrow gap is a demanding test of agile flight, requiring centimeter-level spatial alignment and precise temporal execution. We address this challenge using a control policy that operates directly on onboard visual and inertial measurements to output collective thrust and body-rate commands. The task is difficult because success lies on a thin feasible manifold: small errors in timing, position, attitude, or visual observability cause collision. Prior planning and learning methods rely on privileged state, teachers, planners, or distillation; learning …
Manipulation policies are evaluated predominantly by binary task success, which cannot tell whether the commanded gesture is the one the object's kinematics admit. We introduce Constraint-Referenced Strategy (CRS), which projects the commanded end-effector twist onto the object's joint axis, separating progress along the joint from off-constraint waste. CRS requires a joint model and a joint-following scripted controller derived from it, but no recorded demonstration or learned judge. On 74 simulated articulated objects, success registers three transfer failures as lower rates but attributes …
Anukriti Singh, Maxime Caniot, Pratap Tokekar, Firas Abi-Farraj
Abstract摘要
Imitation policies that reliably execute most of a manipulation trajectory can still fail in localized regions requiring last-mile precision. Updating the full policy or exploring residual actions everywhere treats this local failure as a global learning problem. We introduce Adaptive Residual Correction (ARC), a lightweight post-training module that uses reinforcement learning (RL) to learn a residual correction on top of a frozen policy while using sparse human corrections to structure autonomous exploration. Rather than treating human corrections as imitation targets, ARC uses …
@misc{singh2026when,
title = {{When, Where, and What to Correct: Human-Guided Adaptive Residual Correction for Robot Policies}},
author = {Anukriti Singh and Maxime Caniot and Pratap Tokekar and Firas Abi-Farraj},
year = {2026},
eprint = {2610.37547},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.37547},
}