用多点振动数据恢复复杂物体的声音,突破传统视觉麦克风限制。
Hearing the Room Through the Shape of the Drum: Modal-Guided Sound Recovery from Multi-Point Surface Vibrations

- 基于物体振动模态构建物理引导模型,连接声音与多点振动
- 在多种日常物体上实现声音重建,性能显著优于单点方法
- 适合对弱振动或共振物体进行声源还原的研究者
光学振动传感可直接从附近物体表面振动恢复场景声音,使日常物体成为“视觉麦克风”。然而,以往方法主要聚焦于具有优良振动响应的特定物体,如扬声器膜或薄膜类物体(如薯片袋、植物叶片)。本文针对更难处理的固体物体——其振动响应弱或高度共振——提出新方法。通过散斑基振动成像系统同步捕捉物体多个表面点的多轴振动信号,构建一种新的物理引导振动生成模型,该模型通过物体的振动模态将场景声源与多点多轴振动联系起来。利用此模型反演物体的共振传递函数,融合多路振动信号以估计原始声源。我们在多种常见物体上验证该方法,结果表明,在挑战性场景下,其性能显著优于传统单点散斑振动测量,也优于其他多信号融合的信号处理方法。
原文摘要 · Abstract (English)
Optical vibration sensing enables recovering the scene sound directly from the surface vibration of nearby objects, turning everyday objects into ``visual microphones''. However, most prior methods had focused on capturing the vibrations of specific objects with highly favorable vibration responses. These include objects where the surface vibrations are generated by the object itself (e.g., speaker membrane or guitar body) or objects consisting of a thin membrane which is highly reactive to sound (e.g., a chip bag or the leaf of a plant). In this paper, we tackle sound recovery for a more challenging class of solid objects whose vibration responses are poor or highly resonant. We simultaneously capture vibrations for multiple surface points on the object using a speckle-based vibrometry imaging system. Then, we derive a novel physics-guided vibration formation model that relates the scene sound source to the captured multi-point multi-axis vibrations via the object's vibrational modes. The model is then used to reverse the resonant transfer function of the vibrating object, fusing multiple vibration signals to estimate the original sound source in the scene. We evaluate our approach by recovering sound from a variety of everyday objects, demonstrating that it significantly outperforms traditional single-point speckle vibrometry in challenging scenarios and other signal-processing-based methods for multi-signal fusing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。