Microsoft Research · embodied AI & robot learning微软研究院 · 具身智能与机器人学习
5 curated papers, code, datasets and hardware from Microsoft Research (5 with code), with one-line takeaways.5 个来自微软研究院的精选论文、代码、数据集与硬件（5 个附代码），均配一句话要点。
Labs are matched from each entry's affiliation field by simple name rules (data/labs.yaml); joint work appears under every lab named. Only curated entries are listed, because daily arXiv papers carry no affiliations.机构由每个条目的机构字段按名称规则匹配（data/labs.yaml），合作工作会出现在所有署名机构下。仅列出精选条目，因为每日 arXiv 论文不含机构信息。
One model for UI navigation and robot manipulation, pretrained on images, videos and robot data labelled with Set-of-Mark (what to act on) and Trace-of-Mark (how things move); model and code are public.同一个模型同时做界面操作与机器人操作：在图像、视频和机器人数据上预训练，用 Set-of-Mark 标注可操作对象、用 Trace-of-Mark 标注运动轨迹；模型与代码公开。
@misc{yang2025magma,
title = {{Magma: A Foundation Model for Multimodal AI Agents}},
author = {Jianwei Yang and Reuben Tan and Qianhui Wu and Ruijie Zheng and Baolin Peng and Yongyuan Liang and Yu Gu and Mu Cai and Seonghyeon Ye and Joel Jang and Yuquan Deng and Lars Liden and Jianfeng Gao},
year = {2025},
eprint = {2502.13130},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2502.13130},
}
Builds a bimanual VLA by composing two copies of a pretrained single-arm VLA instead of pretraining on bimanual data; reports beating a similar-size RDT-1B and narrowing the gap to π0 on real and sim tasks.不做双臂预训练，而是把两份预训练单臂 VLA 组合成协同的双臂 VLA；论文报告在真实与仿真双臂任务上优于同规模 RDT-1B，并缩小了与依赖大量双臂数据的 π0 的差距。
@misc{im2025twinvla,
title = {{TwinVLA: Data-Efficient Bimanual Manipulation with Twin Single-Arm Vision-Language-Action Models}},
author = {Hokyun Im and Euijin Jeong and Andrey Kolobov and Jianlong Fu and Youngwoon Lee},
year = {2025},
eprint = {2511.05275},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2511.05275},
}
Instead of emitting action tokens from the VLM, a separate diffusion action transformer is conditioned on the VLM's "cognition" feature; reports clear gains over OpenVLA in sim and on real robots.不再让 VLM 直接输出动作 token，而是用独立的扩散动作 Transformer 以 VLM 的“认知”特征为条件生成动作；报告在仿真和真机上明显优于 OpenVLA。
@misc{li2024cogact,
title = {{CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation}},
author = {Qixiu Li and Yaobo Liang and Zeyu Wang and Lin Luo and Xi Chen and Mozheng Liao and Fangyun Wei and Yu Deng and Sicheng Xu and Yizhong Zhang and Xiaofan Wang and Bei Liu and Jianlong Fu and Jianmin Bao and Dong Chen and Yuanchun Shi and Jiaolong Yang and Baining Guo},
year = {2024},
eprint = {2411.19650},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2411.19650},
}
LAPA pretrains a VLA without action labels: it learns discrete latent actions between video frames, predicts them from image + text, then maps them to robot actions; reports beating OpenVLA on real tasks.LAPA 无需动作标注即可预训练 VLA：先从视频相邻帧间学习离散隐动作，再训练模型由图像和指令预测隐动作，最后用少量机器人数据映射到真实动作；报告在真机任务上优于 OpenVLA。
@misc{ye2024latent,
title = {{Latent Action Pretraining from Videos}},
author = {Seonghyeon Ye and Joel Jang and Byeongguk Jeon and Sejune Joo and Jianwei Yang and Baolin Peng and Ajay Mandlekar and Reuben Tan and Yu-Wei Chao and Bill Yuchen Lin and Lars Liden and Kimin Lee and Jianfeng Gao and Luke Zettlemoyer and Dieter Fox and Minjoon Seo},
year = {2024},
eprint = {2410.11758},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2410.11758},
}
Draws the robot's recent motion traces on the input image ('visual trace prompting') and fine-tunes OpenVLA on 150K trajectories; reports ~10% gains on SimplerEnv and 3.5x on real WidowX tasks; 4B variant too.把机器人近期的运动轨迹画在输入图像上（“视觉轨迹提示”），并在 15 万条轨迹上微调 OpenVLA；报告在 SimplerEnv 上提升约 10%、真实 WidowX 任务上提升 3.5 倍，另有基于 Phi-3-Vision 的 4B 精简版。
@misc{zheng2024tracevla,
title = {{TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies}},
author = {Ruijie Zheng and Yongyuan Liang and Shuaiyi Huang and Jianfeng Gao and Hal Daumé and Andrey Kolobov and Furong Huang and Jianwei Yang},
year = {2024},
eprint = {2412.10345},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2412.10345},
}