无需训练，用记忆回放实现机器人实时操作
RT-Cache: Training-Free Retrieval for Real-Time Manipulation
- 将图像动作轨迹存入向量内存，测试时通过当前帧检索并重播多步动作
- 在百万级数据下查询耗时低于1秒，成功率提升约2倍，速度更快30%
- 适合快速部署、少样本适应复杂任务，可与高层策略结合使用
真实机器人需在新环境中以极少新数据重复行为，但现有控制器或每步推理开销大，或需部署时微调。我们提出RT-Cache，一种无需训练的检索式控制框架：将多样图像-动作轨迹存入统一向量内存，在测试时对当前帧嵌入并检索、重播多步动作片段，替代每步模型调用。分层搜索使百万级数据下的查询耗时低于1秒，将成本从计算转向存储，实现在中等显卡上实时控制。在真实机器人任务和大型公开日志上，RT-Cache成功率比强基线高约2倍，完成时间快30%；单次演示锚定实验显示，无需微调即可立即适应更复杂的接触密集型任务。RT-Cache将经验转为可追加的内存，为今日少样本部署提供简单可扩展路径，并为多模态键和高层策略集成奠定基础。
原文摘要 · Abstract (English)
Real robots are expected to repeat the same behavior in new environments with very little new data, yet modern controllers either incur heavy per-step inference or require deployment-time fine-tuning. We propose RT-Cache, a training-free retrieval-as-control pipeline that caches diverse image action trajectories in a unified vector memory and, at test time, embeds the current frame to retrieve and replay multi-step snippets, replacing per-step model calls. A hierarchical search keeps lookups sub-second at million scale, shifting cost from compute to storage and enabling real-time control on modest GPUs. Across real-robot tasks and large open logs, RT-Cache achieves higher success and lower completion time than strong retrieval baselines (approximately x2 higher success and ~30% faster in our settings), and a single-episode anchoring study shows immediate adaptation to a more complex, contact-rich task without fine-tuning. RT-Cache turns experience into an append-only memory, offering a simple, scalable path to few-shot deployment today and a foundation for multimodal keys and optional integration with high-level policies. Project page: https://rt-cache.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。