arXiv:2412.19078eess.ASeess.SP2024-12中稿 · the IEEE Internati…被引 1

用图注意力融合声学特征,提升交通车辆识别与方向判断精度。

Graph-Enhanced Dual-Stream Feature Fusion with Pre-Trained Model for Acoustic Traffic Monitoring

  • 双流结构分别提取车型和方向特征,再通过图注意力融合。
  • 在DCASE 2024挑战赛中取得第一名,性能显著优于基线。
  • 适合音频监控、智慧交通等需细粒度声源分析的场景。

麦克风阵列广泛应用于声源定位与智慧城市声学交通监测,但受限于真实交通音频数据标注稀缺及场景复杂多变。DCASE 2024挑战赛任务10旨在利用多通道音频信号实现车辆计数(小汽车或商用车)并识别行驶方向(左至右或反之)。本文提出一种图增强双流特征融合网络(GEDF-Net),同时考虑车型与方向信息以提升检测效果。方法包含车型特征提取(VTFE)分支、方向特征提取(VDFE)分支及帧级特征融合模块。在VTFE分支中引入预训练模型PANNs缓解数据稀缺问题,并通过图注意力机制挖掘时序关系,突出关键音频事件。帧级融合使特征表示更精细,最终在测试集上表现优异。本方法为团队在DCASE 2024任务10中的提交方案,获得第一名。

原文摘要 · Abstract (English)

Microphone array techniques are widely used in sound source localization and smart city acoustic-based traffic monitoring, but these applications face significant challenges due to the scarcity of labeled real-world traffic audio data and the complexity and diversity of application scenarios. The DCASE Challenge's Task 10 focuses on using multi-channel audio signals to count vehicles (cars or commercial vehicles) and identify their directions (left-to-right or vice versa). In this paper, we propose a graph-enhanced dual-stream feature fusion network (GEDF-Net) for acoustic traffic monitoring, which simultaneously considers vehicle type and direction to improve detection. We propose a graph-enhanced dual-stream feature fusion strategy which consists of a vehicle type feature extraction (VTFE) branch, a vehicle direction feature extraction (VDFE) branch, and a frame-level feature fusion module to combine the type and direction feature for enhanced performance. A pre-trained model (PANNs) is used in the VTFE branch to mitigate data scarcity and enhance the type features, followed by a graph attention mechanism to exploit temporal relationships and highlight important audio events within these features. The frame-level fusion of direction and type features enables fine-grained feature representation, resulting in better detection performance. Experiments demonstrate the effectiveness of our proposed method. GEDF-Net is our submission that achieved 1st place in the DCASE 2024 Challenge Task 10.

声学监测图神经网络多模态融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。