arXiv:2507.02755eess.AScs.AI2025-07

多智能体并行处理声音场景,提升响应速度与抗错能力。

Multi-agent Auditory Scene Analysis

  • 任务并行运行,通过反馈机制修正局部错误
  • 定位误差可由分离质量反馈修正,分类结果降低干扰敏感度
  • 轻量级设计适合听觉辅助、人机交互等实时场景

听觉场景分析(ASA)旨在从声学环境中提取信息,主要包括声源定位、分离和分类三项任务。传统方法采用线性流程:先定位声源,再基于位置分离出各音频流,最后从中提取应用相关特征(如事件检测、说话人识别、情绪分类等)。该流程导致整体响应时间长,且后继任务对定位误差高度敏感。现有技术虽大幅降低误差,但计算开销大,难以满足生物声学、助听器、搜救、人机交互等对低延迟和小算力需求的应用。为此,本文提出多智能体听觉场景分析(MASA)系统,将三项任务并行执行,并引入智能体间的反馈回路以补偿局部错误:利用分离输出质量修正定位偏差;利用分类结果降低定位对干扰的敏感性。所提系统在不显著增加复杂度的前提下,实现强鲁棒性和低延迟。完整框架基于开源工具(JACK用于音频采集/播放,ROS2用于智能体通信),支持用户自定义扩展。

原文摘要 · Abstract (English)

Auditory scene analysis (ASA) aims to retrieve information from the acoustic environment, by carrying out three main tasks: sound source location, separation, and classification. These tasks are traditionally executed with a linear data flow, where the sound sources are first located; then, using their location, each source is separated into its own audio stream; from each of which, information is extracted that is relevant to the application scenario (audio event detection, speaker identification, emotion classification, etc.). However, running these tasks linearly increases the overall response time, while making the last tasks (separation and classification) highly sensitive to errors of the first task (location). A considerable amount of effort and computational complexity has been employed in the state-of-the-art to develop techniques that are the least error-prone possible. However, doing so gives rise to an ASA system that is non-viable in many applications that require a small computational footprint and a low response time, such as bioacoustics, hearing-aid design, search and rescue, human-robot interaction, etc. To this effect, in this work, a multi-agent approach is proposed to carry out ASA where the tasks are run in parallel, with feedback loops between them to compensate for local errors, such as: using the quality of the separation output to correct the location error; and using the classification result to reduce the localization's sensitivity towards interferences. The result is a multi-agent auditory scene analysis (MASA) system that is robust against local errors, without a considerable increase in complexity, and with a low response time. The complete proposed MASA system is provided as a publicly available framework that uses open-source tools for sound acquisition and reproduction (JACK) and inter-agent communication (ROS2), allowing users to add their own agents.

听觉分析多智能体实时系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。