开源工具可3D可视化鸟类鸣叫,支持分析与合成。
SoundPlot: An Open-Source Framework for Birdsong Acoustic Analysis and Neural Synthesis with Interactive 3D Visualization
- 提取频谱、音高、梅尔系数等特征,构建统一声学空间。
- 合成音频与原音频谱相关性超0.92,保留听觉结构。
- 适合生物声学、音频处理与计算行为学研究者使用。
我们提出SoundPlot,一个用于鸟类鸣叫声学分析与神经合成的开源框架。系统通过提取频谱特征(质心、带宽、对比度)、基于概率YIN(pYIN)的音高轮廓及梅尔频率倒谱系数(MFCCs),将音频信号映射至统一的音色空间,实现基于Web的实时3D动态可视化。音频重建采用格里芬-林相位估计算法处理梅尔频谱图。配套的Three.js界面提供双视口对比,分别显示原始与合成音频轨迹,并支持独立播放控制。通过波形分析、频谱比较和主成分分析(PCA)评估特征空间表现。定量结果显示,合成音频与原始音频的梅尔频谱相关性超过0.92,表明感知声学结构高度保真。SoundPlot以MIT许可证发布,旨在推动生物声学、音频信号处理与计算行为学研究。
原文摘要 · Abstract (English)
We present SoundPlot, an open-source framework for analyzing avian vocalizations through acoustic feature extraction, dimensionality reduction, and neural audio synthesis. The system transforms audio signals into a multi-dimensional acoustic feature space, enabling real-time visualization of temporal dynamics in 3D using web-based interactive graphics. Our framework implements a complete analysis-synthesis pipeline that extracts spectral features (centroid, bandwidth, contrast), pitch contours via probabilistic YIN (pYIN), and mel-frequency cepstral coefficients (MFCCs), mapping them to a unified timbre space for visualization. Audio reconstruction employs the Griffin-Lim phase estimation algorithm applied to mel spectrograms. The accompanying Three.js-based interface provides dual-viewport visualization comparing original and synthesized audio trajectories with independent playback controls. We demonstrate the framework's capabilities through comprehensive waveform analysis, spectrogram comparisons, and feature space evaluation using Principal Component Analysis (PCA). Quantitative evaluation shows mel spectrogram correlation scores exceeding 0.92, indicating high-fidelity preservation of perceptual acoustic structure. SoundPlot is released under the MIT License to facilitate research in bioacoustics, audio signal processing, and computational ethology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。