让视觉、语言、动作各自按频率更新，提升机器人操作成功率
DAM-VLA: Decoupled Asynchronous Multimodal Vision Language Action model
- 每种模态独立按传感器频率更新，避免同步处理的资源浪费
- 在7个真实场景任务中，成功率超基线两倍以上（95.2% vs 40.95%）
- 支持100Hz实时响应，适合高精度物理交互场景
视觉-语言-动作（VLA）模型继承自视觉-语言预训练的统一时钟，以固定速率处理所有输入。这与物理交互不匹配：高频模态变化可达百赫兹，视觉演化较慢，语言在任务中保持不变。同步式VLA会过度采样慢模态，忽略快模态，且动作生成受限于最低有效频率。我们提出解耦时序处理的DAM-VLA，为每类模态维护独立的隐状态缓冲区，按传感器频率刷新，并由动作头持续读取。通过门控交叉注意力融合新高频信号，同时保留预训练主干网络。在七个接触密集的真实世界操作任务中，DAM-VLA平均成功率超过最强同步基线两倍（95.2% vs 40.95%），并实现稳定、响应迅速的100 Hz控制。
原文摘要 · Abstract (English)
Vision-language-action (VLA) models inherit a shared synchronous clock from vision-language pretraining, processing every input at one rate. This is misaligned with physical interaction, where a high-frequency modality changes at hundreds of hertz, vision evolves more slowly, and language stays constant across an episode. A synchronous VLA oversamples slow modalities, undersamples fast ones, and caps action generation at the lowest effective frequency. We hypothesize that decoupling temporal processing per modality, letting each update and retain information at its own sensor rate, yields stronger representations and more robust control. We present DAM-VLA, which maintains per-modality latent buffers refreshed at sensor rates and read continuously by the action head, integrating new high-frequency modalities through gated cross-attention that leaves the pretrained backbone intact. Across seven contact-rich real-world manipulation tasks, DAM-VLA more than doubles the average success rate of the strongest synchronous baseline (95.2\% vs.\ 40.95\%) while sustaining smooth, reactive 100\,Hz control. Project website: \href{https://intuitive-robots.github.io/DAM-VLA/}{intuitive-robots.github.io/DAM-VLA/}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。