用事件数据提升机器人在暗光模糊环境下的操作鲁棒性
E-VLA: Event-Augmented Vision-Language-Action Model for Dark and Blurred Scenes

- 直接利用事件流中的运动和结构信息,不重建图像
- 暗光下成功率从0%提升至90%,严重模糊下从0%到32.5%
- 适合做低光照、高速运动场景的机器人视觉系统
机器人视觉-语言-动作(VLA)模型在开放任务中泛化能力强,但在极端低光、运动模糊和黑裁切等感知退化条件下表现脆弱。本文提出E-VLA,一种事件增强的VLA框架,在传统帧基视觉不可靠时提升操作鲁棒性。E-VLA不重建图像,而是直接利用事件流中的运动与结构线索,保持语义感知与感知-动作一致性。我们构建了基于DAVIS346事件相机的开源遥操作平台,采集了跨多种任务与光照的真实世界同步RGB-事件-动作数据集。提出轻量级、可预训练的事件融合策略,并研究事件窗口化以实现稳定部署。实验表明,仅简单叠加累积事件图至RGB图像,即可显著提升暗光与强模糊场景下的性能:在20流明的拾取-放置任务中,成功率从0%(仅图像)提升至60%(叠加融合),再增至90%(事件适配器);在1000毫秒曝光模拟的严重运动模糊下,拾取-放置成功率从0%升至20-25%,分拣任务从5%升至32.5%。整体证明事件驱动感知可有效融入VLA模型,推动超越传统帧基成像的鲁棒具身智能。代码与数据集将公开于https://github.com/JJayzee/E-VLA。
原文摘要 · Abstract (English)
Robotic Vision-Language-Action (VLA) models generalize well for open-ended manipulation, but their perception is fragile under sensing-stage degradations such as extreme low light, motion blur, and black clipping. We present E-VLA, an event-augmented VLA framework that improves manipulation robustness when conventional frame-based vision becomes unreliable. Instead of reconstructing images from events, E-VLA directly leverages motion and structural cues in event streams to preserve semantic perception and perception-action consistency under adverse conditions. We build an open-source teleoperation platform with a DAVIS346 event camera and collect a real-world synchronized RGB-event-action manipulation dataset across diverse tasks and illuminations. We also propose lightweight, pretrained-compatible event integration strategies and study event windowing for stable deployment. Experiments show that even a simple parameter-free fusion, i.e., overlaying accumulated event maps onto RGB images, could substantially improve robustness in dark and heavy-blur scenes: on Pick-Place at 20 lux, success increases from 0% (image-only) to 60% with overlay fusion and to 90% with our event adapter; under severe motion blur (1000 ms-exposure proxy), Pick-Place improves from 0% to 20-25%, and Sorting from 5% to 32.5%. Overall, E-VLA provides systematic evidence that event-driven perception can be effectively integrated into VLA models, pointing toward robust embodied intelligence beyond conventional frame-based imaging. Code and dataset will be available at https://github.com/JJayzee/E-VLA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。