让大模型精准定位音频中短事件发生时间
SpotSound: Enhancing Large Audio-Language Models with Fine-Grained Temporal Grounding
- 设计新训练目标，抑制无事件时的虚假时间定位
- 在事件占比不足10%的复杂音频中仍达顶尖定位精度
- 适合需要高精度时间定位的语音分析与媒体检索场景
大型音频语言模型（ALMs）在整体音频理解方面表现出色，但在时间定位任务上仍不可靠，即难以精确定位长音频中事件发生的具体时刻。这一问题源于两个因素：训练数据以片段级监督为主，缺乏精确时间戳；评估基准未能模拟真实场景中短事件被密集背景音掩盖的情况。本文提出SpotSound，一种面向音频事件定位的音频语言模型。SpotSound引入新型训练目标，专门抑制输入中不存在事件时的幻觉时间定位。此外，我们构建了SpotSound-Bench，一个具有挑战性的时序定位基准，其中目标事件在每段音频中占比不足约10%，形成严格的‘大海捞针’式评测。实验表明，SpotSound在时序定位基准上达到当前最优性能，同时在通用下游音频语言任务中保持稳健表现。代码、模型与基准已公开于https://loiesun.github.io/spotsound/
原文摘要 · Abstract (English)
Large Audio-Language Models (ALMs) have recently demonstrated remarkable capabilities in holistic audio understanding, yet they remain unreliable for temporal grounding, i.e., the task of pinpointing exactly when an event occurs within long-form audio. This limitation stems from two factors: training data dominated by clip-level supervision lacking precise timestamps, and benchmarks that fail to simulate real-world scenarios where short events are obscured by dense background sounds. In this paper, we introduce SpotSound, an audio language model designed for grounding audio events. SpotSound incorporates a novel training objective, specifically designed to suppress hallucinated timestamps for events absent from the input. Additionally, we present SpotSound-Bench, a challenging temporal grounding benchmark where target events occupy less than ~10% of each clip, creating a rigorous 'needle-in-a-haystack' evaluation. Experiments demonstrate that SpotSound achieves state-of-the-art results on temporal grounding benchmarks while maintaining robust performance across general downstream audio-language tasks. Code, models and benchmark are released on https://loiesun.github.io/spotsound/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。