让语音模型在测试时自动适应噪声环境，无需额外数据。
SLM-TTA: A Framework for Test-Time Adaptation of Generative Spoken Language Models
- 测试时仅更新少量参数，用当前语音自适应调整模型。
- 在19项音频任务上提升抗干扰能力，准确率不下降。
- 轻量高效，适合手机等低算力设备部署。
生成式语音模型（SLMs）在语音驱动应用中日益重要，但在真实噪声、混响和麦克风差异下性能下降。现有方法依赖离线域适应，耗时且需大量数据。本文提出首个针对生成式SLMs的测试时自适应（TTA）框架，可处理音视频交替输入。该方法在推理时仅基于当前语音片段，通过更新极小部分参数实现自适应，无需源数据或标签。此策略稳定了词元分布，增强对声学变化的鲁棒性，同时保持核心任务精度。在AIR-Bench的19个音频理解任务上，该方法在多种噪声条件下均取得一致提升。因仅修改少量权重，该方案计算与内存开销极低，适用于资源受限平台。本工作显著提升了生成式SLMs在真实场景中的适应能力。
原文摘要 · Abstract (English)
Spoken Language Models (SLMs) are increasingly central to modern speech-driven applications, but performance degrades under acoustic shift - real-world noise, reverberation, and microphone variation. Prior solutions rely on offline domain adaptation, which is post-hoc, data-intensive, and slow. We introduce the first test-time adaptation (TTA) framework for generative SLMs that process interleaved audio-text prompts. Our method updates a small, targeted subset of parameters during inference using only the incoming utterance, requiring no source data or labels. This stabilizes token distributions and improves robustness to acoustic variability without degrading core task accuracy. Evaluated on automatic speech recognition, speech translation, and 19 audio understanding tasks from AIR-Bench, our approach yields consistent gains under diverse corruptions. Because adaptation touches only a small fraction of weights, it is both compute- and memory-efficient, supporting deployment on resource-constrained platforms. This work enhances the robustness and adaptability of generative SLMs for real-world speech-driven applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。