arXiv:2604.15383cs.SDcs.AI2026-04ACL被引 2

让大模型更准确地捕捉声音细节,不依赖训练的解码新方法

Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models

论文配图:Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models
图 1 · 摘自论文原文
  • 通过对比原始音频与模糊版本的解码结果,动态调整输出
  • 在MMAU和AIR-Bench上提升统一音视频模型性能
  • 无需训练,适配各类大型音视频模型,特别适合追求细节的场景

大型音视频模型(LALMs)能泛化于语音、声响与音乐,但统一解码器可能产生时间平滑偏差:瞬时声学线索易被忽视,而更平滑的上下文因语言先验支持更强而占主导,导致输出不够精准。本文提出训练无关的时序对比解码(Temporal Contrastive Decoding, TCD),在推理阶段缓解此问题。TCD通过平滑输入波形并重新编码生成时序模糊的慢路径视图,对比原输入与慢路径视图的下一个词元概率分布。该对比信号以受限候选集内的逐词元概率修正形式应用,自归一化稳定度分数确定模糊窗口与更新尺度,基于不确定性和音频依赖性的分步门控仅在必要时激活更新。在MMAU与AIR-Bench上的实验表明,TCD对强统一音视频模型均带来一致性能提升。我们还进行了消融分析与架构适用性研究,揭示关键组件贡献及TCD在不同大型音视频模型设计中的行为特性。

原文摘要 · Abstract (English)

Large audio-language models (LALMs) generalize across speech, sound, and music, but unified decoders can exhibit a \emph{temporal smoothing bias}: transient acoustic cues may be underutilized in favor of temporally smooth context that is better supported by language priors, leading to less specific audio-grounded outputs. We propose \emph{Temporal Contrastive Decoding} (TCD), a training-free decoding method for unified LALMs that mitigates this effect at inference time. TCD constructs a temporally blurred slow-path view by smoothing the input waveform and re-encoding it, then contrasts next-token logits from the original and slow-path views. The contrastive signal is applied as a token-level logit update restricted to a small candidate set. A self-normalized stability score sets the blur window and update scale, and a step-wise gate based on uncertainty and audio reliance activates the update only when needed. Experiments on MMAU and AIR-Bench show consistent improvements on strong unified LALMs. We further conduct ablations and an architectural applicability study to analyze the contributions of key components and how TCD behaves across large audio-language model designs.

音视频模型解码优化对比学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。