arXiv:2603.09385cs.CV2026-03中稿 · ECCV被引 4

通过时空蒸馏提升事件相机深度估计一致性,显著减少误差。

EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation

  • 将事件流视为连续视频,引入跨模态蒸馏策略
  • 在EventScape上30米处深度误差降低53%(2.30→1.06)
  • 适用于高动态、高速运动场景下的鲁棒3D感知

事件相机对高速运动和极端光照具有优异敏感性,使基于事件的单目深度估计成为挑战条件下鲁棒三维感知的有前景方法。然而,密集深度标注的稀缺严重阻碍了进展。尽管近期无标注方法通过从视觉基础模型(VFMs)中蒸馏知识缓解了这一问题,但其关键局限在于将事件流处理为独立帧,忽略了事件数据固有的时间连续性,导致未能利用视觉基础模型中的丰富时空先验,最终产生时间不一致且精度较低的深度预测。为此,我们提出EventVGGT,一个首次在事件域中显式建模时空与多视角几何先验的框架。通过三层次蒸馏策略实现:(i) 跨模态特征混合(CMFM)在输出层融合RGB与事件特征生成辅助深度预测;(ii) 空间-时间特征蒸馏(STFD)在特征层蒸馏VGGT强大的时空表示;(iii) 时间一致性蒸馏(TCD)在时间层对齐帧间深度变化以增强一致性。大量实验表明,EventVGGT持续优于现有方法——在EventScape上30米处绝对平均深度误差降低超过53%(从2.30降至1.06),并在未见的DENSE和MVSEC数据集上表现出稳健的零样本泛化能力。代码已公开于https://github.com/yinruiRen/EventVGGT。

原文摘要 · Abstract (English)

Event cameras offer superior sensitivity to high-speed motion and extreme lighting, making event-based monocular depth estimation a promising approach for robust 3D perception in challenging conditions. However, progress is severely hindered by the scarcity of dense depth annotations. While recent annotation-free approaches mitigate this by distilling knowledge from Vision Foundation Models (VFMs), a critical limitation persists: they process event streams as independent frames. By neglecting the inherent temporal continuity of event data, these methods fail to leverage the rich temporal priors encoded in VFMs, ultimately yielding temporally inconsistent and less accurate depth predictions. To address this, we introduce EventVGGT, a novel framework that explicitly models the event stream as a coherent video sequence. To the best of our knowledge, we are the first to distill spatio-temporal and multi-view geometric priors from the Visual Geometry Grounded Transformer (VGGT) into the event domain. We achieve this via a comprehensive tri-level distillation strategy: (i) Cross-Modal Feature Mixture (CMFM) bridges the modality gap at the output level by fusing RGB and event features to generate auxiliary depth predictions; (ii) Spatio-Temporal Feature Distillation (STFD) distills VGGT's powerful spatio-temporal representations at the feature level; and (iii) Temporal Consistency Distillation (TCD) enforces cross-frame coherence at the temporal level by aligning inter-frame depth changes. Extensive experiments demonstrate that EventVGGT consistently outperforms existing methods -- reducing the absolute mean depth error at 30m by over 53\% on EventScape (from 2.30 to 1.06) -- while exhibiting robust zero-shot generalization on the unseen DENSE and MVSEC datasets. The code is available at https://github.com/yinruiRen/EventVGGT.

事件相机深度估计蒸馏时空建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。