仅凭倒液声音，就能推断液体量、容器形状和倒速等物理属性。
The Sound of Water: Inferring Physical Properties from Pouring Liquids
- 利用声音基频推断液体物理特性，理论可行。
- 模型在真实数据上准确预测液位、容器形状等参数。
- 适用于多种容器和真实场景，适合机器人感知研究。
我们研究声音与视觉观察之间的关系，以及其与日常活动——倒液体——背后物理规律的联系。仅凭液体倒入容器时的声音，目标是自动推断液体高度、容器形状与尺寸、倒液速率及填满时间等物理属性。为此，我们：(i) 理论证明这些属性可通过基频（音高）确定；(ii) 使用模拟数据和视觉数据，结合物理启发的目标函数训练音高检测模型；(iii) 构建首个大规模真实倒液视频数据集，支持系统性研究；(iv) 验证训练模型可准确推断真实数据中的物理属性；(v) 展示模型对多种容器形状、其他数据集及YouTube真实视频具有强泛化能力。本工作揭示了声学、物理与学习交叉领域的深层洞察，为增强机器人倒液任务的多感官感知提供了新可能。
原文摘要 · Abstract (English)
We study the connection between audio-visual observations and the underlying physics of a mundane yet intriguing everyday activity: pouring liquids. Given only the sound of liquid pouring into a container, our objective is to automatically infer physical properties such as the liquid level, the shape and size of the container, the pouring rate and the time to fill. To this end, we: (i) show in theory that these properties can be determined from the fundamental frequency (pitch); (ii) train a pitch detection model with supervision from simulated data and visual data with a physics-inspired objective; (iii) introduce a new large dataset of real pouring videos for a systematic study; (iv) show that the trained model can indeed infer these physical properties for real data; and finally, (v) we demonstrate strong generalization to various container shapes, other datasets, and in-the-wild YouTube videos. Our work presents a keen understanding of a narrow yet rich problem at the intersection of acoustics, physics, and learning. It opens up applications to enhance multisensory perception in robotic pouring.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。