$R^3$: Training Robots to Reason in Natural Language via Reinforcement Learning
Post-training recipe that turns an off-the-shelf VLM into a free-form language reasoner guiding a low-level policy: mid-train on expert reasoning traces, then rubric-based RL from offline action data.把现成 VLM 训练成用自由语言推理来指导底层策略的“机器人推理器”：先用专家推理轨迹做中训练，再用离线动作数据做基于评分规则的单步强化学习。
BibTeX
@misc{wu2026r,
title = {{$R^3$: Training Robots to Reason in Natural Language via Reinforcement Learning}},
author = {Lehong Wu and Yuxiao Qu and Zheyuan Hu and Ivan Zhang and Limin Wei and Zackory Erickson and Aviral Kumar},
year = {2026},
eprint = {2608.26053},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2608.26053},
}