用事件相机从微小振动中还原音频，速度快且抗干扰。
Event2Audio: Event-Based Optical Vibration Sensing
- 用事件相机捕捉高速振动信号，提升响应效率。
- 可同时还原多个声源的音频，重建质量达当前最优。
- 适合实时音频恢复与复杂环境下的振动感知应用。
视频中的微小振动能揭示声音、材料特性等视觉之外的信息。当振动可见时可被动记录，不可见时则可用激光主动增强其视觉表现。本文改进了主动感知方法，利用事件相机高效捕捉快速运动。实验表明，该方法即使在存在环境干扰的情况下，仍可从振动中恢复音频，支持多个声源同时处理。相比现有技术，重建质量相当但速度更快，接近实时处理。
原文摘要 · Abstract (English)
Small vibrations observed in video can unveil information beyond what is visual, such as sound and material properties. It is possible to passively record these vibrations when they are visually perceptible, or actively amplify their visual contribution with a laser beam when they are not perceptible. In this paper, we improve upon the active sensing approach by leveraging event-based cameras, which are designed to efficiently capture fast motion. We demonstrate our method experimentally by recovering audio from vibrations, even for multiple simultaneous sources, and in the presence of environmental distortions. Our approach matches the state-of-the-art reconstruction quality at much faster speeds, approaching real-time processing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。