用3D高斯表示生成位置感知的物理音效，让虚拟物体能发出真实撞击声。
SonicGauss: Position-Aware Physical Sound Synthesis for 3D Gaussian Representations
- 结合扩散模型与点变换器，从高斯椭球中提取材质和空间声学特征
- 支持不同撞击位置产生差异化的音效，真实感强
- 适用于多种物体类别，适合交互式3D场景音效生成
尽管3D高斯表示（3DGS）在建模物体几何与外观方面表现优异，但其对声音等物理属性的捕捉仍基本未被探索。本文提出SonicGauss框架，通过利用3DGS固有的几何与材质特性，实现从3DGS生成冲击音效。具体地，我们集成基于扩散的音效生成模型与基于PointTransformer的特征提取器，直接从高斯椭球中推断材质特征与空间-声学关联。该方法支持根据撞击位置变化生成空间可变的音效响应，并在ObjectFolder数据集和真实录音上实现跨类别泛化。实验表明，所提方法能生成逼真的、位置感知的听觉反馈，展现出良好的鲁棒性与泛化能力，为连接3D视觉表示与交互式音效合成提供了新路径。
原文摘要 · Abstract (English)
While 3D Gaussian representations (3DGS) have proven effective for modeling the geometry and appearance of objects, their potential for capturing other physical attributes-such as sound-remains largely unexplored. In this paper, we present a novel framework dubbed SonicGauss for synthesizing impact sounds from 3DGS representations by leveraging their inherent geometric and material properties. Specifically, we integrate a diffusion-based sound synthesis model with a PointTransformer-based feature extractor to infer material characteristics and spatial-acoustic correlations directly from Gaussian ellipsoids. Our approach supports spatially varying sound responses conditioned on impact locations and generalizes across a wide range of object categories. Experiments on the ObjectFolder dataset and real-world recordings demonstrate that our method produces realistic, position-aware auditory feedback. The results highlight the framework's robustness and generalization ability, offering a promising step toward bridging 3D visual representations and interactive sound synthesis. Project page: https://chunshi.wang/SonicGauss
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。