揭示语音模型幻觉的谱演化规律，发现中型模型易崩溃，大型模型主动压缩信息
From Dispersion to Attraction: Spectral Dynamics of Hallucination Across Whisper Model Scales
- 通过谱敏感性理论分析激活图谱，揭示模型规模引发的相变机制
- 中型模型出现13.4%跨注意力秩坍缩，大型模型自注意力反向压缩秩(-2.34%)
- 为理解大模型幻觉提供新视角，适合关注模型安全与内在机理的研究者
大规模语音识别模型中的幻觉问题构成重大安全风险。本文提出谱敏感性定理，预测深层网络在层间增益与对齐作用下，从信号衰减的分散态（Regime I）向秩-1坍缩的吸引子态（Regime II）发生相变。我们通过对抗压力下对Whisper模型（从Tiny到Large-v3-Turbo）激活图谱的特征值谱分析验证该理论。结果表明：中等规模模型呈现结构解体（Regime I），跨注意力秩下降13.4%；而大型模型进入压缩寻求吸引子状态（Regime II），自注意力主动压缩秩（-2.34%），并硬化谱斜率，使模型脱离声学证据。
原文摘要 · Abstract (English)
Hallucinations in large ASR models present a critical safety risk. In this work, we propose the \textit{Spectral Sensitivity Theorem}, which predicts a phase transition in deep networks from a dispersive regime (signal decay) to an attractor regime (rank-1 collapse) governed by layer-wise gain and alignment. We validate this theory by analyzing the eigenspectra of activation graphs in Whisper models (Tiny to Large-v3-Turbo) under adversarial stress. Our results confirm the theoretical prediction: intermediate models exhibit \textit{Structural Disintegration} (Regime I), characterized by a $13.4\%$ collapse in Cross-Attention rank. Conversely, large models enter a \textit{Compression-Seeking Attractor} state (Regime II), where Self-Attention actively compresses rank ($-2.34\%$) and hardens the spectral slope, decoupling the model from acoustic evidence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。