用图注意力融合声学特征,提升交通车辆识别与方向判断精度。
Graph-Enhanced Dual-Stream Feature Fusion with Pre-Trained Model for Acoustic Traffic Monitoring
- 双流结构分别提取车型和方向特征,再通过图注意力融合。
- 在DCASE 2024挑战赛中取得第一名,性能显著优于基线。
- 适合音频监控、智慧交通等需细粒度声源分析的场景。
麦克风阵列广泛应用于声源定位与智慧城市声学交通监测,但受限于真实交通音频数据标注稀缺及场景复杂多变。DCASE 2024挑战赛任务10旨在利用多通道音频信号实现车辆计数(小汽车或商用车)并识别行驶方向(左至右或反之)。本文提出一种图增强双流特征融合网络(GEDF-Net),同时考虑车型与方向信息以提升检测效果。方法包含车型特征提取(VTFE)分支、方向特征提取(VDFE)分支及帧级特征融合模块。在VTFE分支中引入预训练模型PANNs缓解数据稀缺问题,并通过图注意力机制挖掘时序关系,突出关键音频事件。帧级融合使特征表示更精细,最终在测试集上表现优异。本方法为团队在DCASE 2024任务10中的提交方案,获得第一名。
原文摘要 · Abstract (English)
Microphone array techniques are widely used in sound source localization and smart city acoustic-based traffic monitoring, but these applications face significant challenges due to the scarcity of labeled real-world traffic audio data and the complexity and diversity of application scenarios. The DCASE Challenge's Task 10 focuses on using multi-channel audio signals to count vehicles (cars or commercial vehicles) and identify their directions (left-to-right or vice versa). In this paper, we propose a graph-enhanced dual-stream feature fusion network (GEDF-Net) for acoustic traffic monitoring, which simultaneously considers vehicle type and direction to improve detection. We propose a graph-enhanced dual-stream feature fusion strategy which consists of a vehicle type feature extraction (VTFE) branch, a vehicle direction feature extraction (VDFE) branch, and a frame-level feature fusion module to combine the type and direction feature for enhanced performance. A pre-trained model (PANNs) is used in the VTFE branch to mitigate data scarcity and enhance the type features, followed by a graph attention mechanism to exploit temporal relationships and highlight important audio events within these features. The frame-level fusion of direction and type features enables fine-grained feature representation, resulting in better detection performance. Experiments demonstrate the effectiveness of our proposed method. GEDF-Net is our submission that achieved 1st place in the DCASE 2024 Challenge Task 10.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。