用测试数据实时优化音频模型，无需标注就能自我进化。
AQA-TTRL: Self-Adaptation in Audio Question Answering with Test-Time Reinforcement Learning
- 通过多数投票生成伪标签，结合强化学习动态优化模型。
- 在三个数据集上分别提升4.42%和11.04%，3B模型超越未适配的7B模型。
- 适合需要持续适应真实音频场景的部署系统，尤其资源受限时。
大音频语言模型在通用音频理解上表现强大，但部署后无法自适应，限制了其在真实场景的应用。由于监督微调成本高，本文提出AQA-TTRL框架，仅使用无标签测试数据，在测试时通过强化学习实现模型的在线演化。该方法通过多数投票生成伪标签，并引入置信度加权来缓解自生成标签的噪声问题。此外，多轮采样策略有效防止优势崩溃，稳定训练过程。在MMAU、MMAR和MMSU三个数据集上，AQA-TTRL对Qwen2.5-Omni 7B模型带来4.42%的平均性能提升，对3B模型提升达11.04%。值得注意的是，经适配的3B模型性能已超过未经适配的7B模型，验证了测试时自适应在音频理解中的有效性。
原文摘要 · Abstract (English)
Large Audio Language Models (LALMs) exhibit strong capabilities in general audio understanding but remain static after deployment, limiting their adaptability to real-world data. Since supervised fine-tuning is costly, we propose AQA-TTRL, a novel framework for audio understanding that enables on-the-fly evolution via test-time reinforcement learning using only unlabeled test data. It generates pseudo-labels via majority voting and optimizes the model through reinforcement learning. To address the noise in self-generated labels, we introduce confidence weighting to adjust training signals. Furthermore, multiple-attempt sampling mitigates advantage collapse and stabilizes training. Across MMAU, MMAR, and MMSU, AQA-TTRL achieves significant average improvements of 4.42% for Qwen2.5-Omni 7B and 11.04% for the 3B model. Notably, the adapted 3B model outperforms direct inference of the unadapted 7B model, highlighting the effectiveness of test-time adaptation in audio understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。