用自动构建数据提升音频事件定位精度，解决大模型听不清时间点的问题。
Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding
- 通过合成片段+多模型伪标签自动构造带时间戳的训练数据
- 在AEGBench上比零样本定位准确率提升73.9%和23.1%（mIoU）
- 适合研究开放词汇音频定位、大模型时序理解的学者
大型音频-语言模型（LALMs）能流畅理解声音，但在定位事件发生时间上精度不足；传统声音事件检测虽有帧级精度，但仅限封闭标签集。二者交界处是开放词汇音频事件定位任务：根据任意自然语言查询预测目标声音事件的时间区间。当前进展受限于数据稀缺——缺乏大规模开放词汇的起止时间标注，而人工标注成本过高。为此，我们提出Auto-AEG，一种可扩展的数据构建流水线，通过自动数据构造与模型微调实现监督。该方法结合程序生成的片段（提供精确时间真值用于冷启动训练），以及真实音频上多模型生成的伪标签（作为强化学习的奖励信号）。在我们发布的独立难度分层基准AEGBench上，该方法相比零样本显著提升73.9%和23.1%的mIoU，且结果可泛化至其他未见的SED及音频定位基准。结果表明，自动构造数据搭配区间感知奖励设计，是扩展LALMs时序定位能力的有效路径。
原文摘要 · Abstract (English)
Large Audio-Language Models (LALMs) reason fluently about sound yet struggle to localize precisely when events occur, while classical Sound Event Detection attains frame-level precision only over a closed label set. At the intersection of these paradigms lies the task of Open-Vocabulary Audio Event Grounding: predicting all time intervals of a target sound event described by an arbitrary natural language query. Progress is bottlenecked by data scarcity: no large-scale resource provides open-vocabulary onset/offset supervision, and manual temporal annotation is prohibitively expensive. To address this, we introduce Auto-AEG, a scalable pipeline that constructs such supervision by automatic data construction and model fine-tuning. It pairs programmatically synthesized clips, which carry placement-exact ground-truth intervals for supervised cold-start, with multi-model pseudo-labels on real-world audio that supply the reward signal for reinforcement learning. Training with this pipeline yields large temporal-localization gains (+73.9% and +23.1% mIoU over zero-shot) on AEGBench, an independent difficulty-stratified benchmark we release, and these gains generalize to held-out SED and other audio grounding benchmarks. Our results show that automatically constructed data, coupled with interval-aware reward design, provides an effective data-side route to expanding the temporal localization capability of LALMs. AEGBench: https://huggingface.co/datasets/zihan-audio/AEGBench
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。