Dita用扩散模型直接生成连续动作,实现跨场景机器人泛化。
Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy
- 用Transformer统一建模视觉、语言与连续动作的扩散过程
- 10次微调即在真实世界完成复杂长程任务,适应环境变化
- 支持多视角、多任务、异构动作空间,轻量开源
尽管近期基于多样化机器人数据集训练的视觉-语言-动作模型在少量领域内数据下展现出良好泛化能力,但其依赖紧凑动作头预测离散或连续动作,限制了对异构动作空间的适应性。我们提出Dita,一个可扩展框架,利用Transformer架构通过统一的多模态扩散过程直接去噪连续动作序列。不同于以往通过浅层网络融合嵌入进行去噪条件建模的方法,Dita采用上下文内条件建模——实现去噪动作与历史观测原始视觉标记之间的细粒度对齐。该设计显式建模动作增量与环境细微差异。通过将扩散动作去噪器与Transformer的可扩展性协同放大,Dita有效整合来自不同机器人形态、相机视角、观察场景、任务和动作空间的跨体感数据集。这种协同增强了对各类变化的鲁棒性,并支持长时序任务的成功执行。在广泛基准测试中表现达到当前最优或具有竞争力。尤其值得注意的是,Dita仅使用第三人称摄像头输入,在10次微调后即可实现对真实环境变化的稳健适应及复杂长程任务的执行。该架构为通用机器人策略学习建立了一个灵活、轻量且开源的基线。项目页面:https://robodita.github.io。
原文摘要 · Abstract (English)
While recent vision-language-action models trained on diverse robot datasets exhibit promising generalization capabilities with limited in-domain data, their reliance on compact action heads to predict discretized or continuous actions constrains adaptability to heterogeneous action spaces. We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences through a unified multimodal diffusion process. Departing from prior methods that condition denoising on fused embeddings via shallow networks, Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations. This design explicitly models action deltas and environmental nuances. By scaling the diffusion action denoiser alongside the Transformer's scalability, Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces. Such synergy enhances robustness against various variances and facilitates the successful execution of long-horizon tasks. Evaluations across extensive benchmarks demonstrate state-of-the-art or comparative performance in simulation. Notably, Dita achieves robust real-world adaptation to environmental variances and complex long-horizon tasks through 10-shot finetuning, using only third-person camera inputs. The architecture establishes a versatile, lightweight and open-source baseline for generalist robot policy learning. Project Page: https://robodita.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。