分离大模型中记忆与上下文模块的更新动态，更准确地衡量不确定性。
DUD: Decoupled Update Dynamics for Reliable Uncertainty Quantification in Large Language Models
- 通过噪声干预分离前馈网络和注意力模块的更新，独立评估其恢复能力。
- 在多个数据集上显著优于现有方法，提升不确定性估计与校准效果。
- 适合关注模型可信度、内部机制分析的研究者使用。
准确的不确定性量化（UQ）对大型语言模型（LLMs）的可靠部署至关重要，但传统基于概率的指标常无法反映模型的真实认知状态。尽管近期机制性方法利用隐藏状态动态，通常会聚合残差流更新，混淆了参数记忆（前馈网络，FFN）与上下文处理（注意力）的不同作用。我们指出这种聚合掩盖了细粒度机制冲突（如记忆-上下文错位），是不确定性的关键信号。为此，提出 extbf{D}ecoupled extbf{U}pdate extbf{D}ynamics extbf{(DUD)} 框架，通过噪声诱导的因果干预显式分离 FFN 与注意力贡献。通过量化各模块的独立恢复能力，构建双流动态特征，捕捉模型内部脆弱性。大量实验表明，DUD 在不确定性估计与校准方面显著优于当前最优基线，并展现出更强的跨数据集泛化能力，验证了解耦动态作为模型忠实度可靠代理的有效性。
原文摘要 · Abstract (English)
Accurate Uncertainty Quantification (UQ) is critical for reliable deployment of Large Language Models (LLMs), yet traditional probability-based metrics often fail to capture the model's true epistemic state. While recent mechanistic approaches leverage hidden state dynamics, they typically aggregate residual stream updates, conflating the distinct roles of parametric memory (Feed-Forward Networks) and contextual processing (Attention). We argue that this aggregation obscures fine-grained mechanistic conflicts, such as memory-context misalignment, that are fundamental indicators of uncertainty. To address this, we introduce \textbf{D}ecoupled \textbf{U}pdate \textbf{D}ynamics \textbf{(DUD)}, a framework that explicitly decouples FFN and Attention contributions via noise-induced causal interventions. By quantifying the independent restoration capabilities of each module, we construct a dual-stream dynamic profile that captures the model's internal fragility. Extensive experiments demonstrate that DUD significantly outperforms state-of-the-art baselines in both uncertainty estimation and calibration, while exhibiting superior cross-dataset generalization, validating decoupled dynamics as a robust proxy for model faithfulness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。