arXiv:2608.11138cs.CLcs.AI2026-08

用注意力路径脆弱性检测大模型预测中的不确定度。

Attention-Path Fragility as an Uncertainty Signal in Large Language Models

论文配图:Attention-Path Fragility as an Uncertainty Signal in Large Language Models
图 1 · 摘自论文原文
  • 提出ASMI,通过遮蔽注意力头并测量子网络间互信息来评估不确定性。
  • 在事实问答任务中,该信号可识别出高置信但易错的预测,误差减少近一半。
  • 无需随机生成,单次贪婪输出即可完成,适合实际部署场景。

我们提出,模型对某个词元的不确定性不仅体现在输出分布的广度上,还体现在其注意力路径受扰动时预测是否脆弱。为此提出ASMI(注意力子网络互信息),一种无需训练的估计器,通过遮蔽注意力头并测量由此产生的子网络间的BALD互信息,结合语义一致核以忽略表面形式的分歧。该信号并非输出置信度的重复:在基于上下文的问答任务中,交叉验证显示它提供了超越单次置信度与熵的错误预测信息,集中在‘高置信但脆弱’的预测上,若据此筛选可使置信过滤保留的误差减半。该信号的有效性具有阶段依赖性,强于答案依赖外部上下文的情况,且受设计限制于仅靠参数知识回忆的情形。Sem-ASMI仅需一次贪婪响应即可读取信号,无需基线所需的随机生成,在十二个基准设置中有十项表现与语义熵持平或更优;在相同十二组设置中,最佳ASMI变体(通常复用基线已采样的十个样本)在八组中持平或领先,三组显著优于基线(配对检验)。在参数化问答任务中,所有变体退化至或低于零成本的MSP基线,完全符合预期,且重运行结果近乎确定。头级分析表明,决定该边界的并非注意力头本身的脆弱性,而是脆弱性是否与错误耦合。

原文摘要 · Abstract (English)

We propose that a model's uncertainty about a token is reflected not only in the breadth of its output distribution but also in whether a confident prediction is \emph{fragile} under perturbation of its attention pathways. We instantiate this as ASMI (Attention-Subnetwork Mutual Information), a training-free estimator that masks attention heads and measures the BALD mutual information among the resulting subnetworks, with a semantic-agreement kernel to discount surface-form disagreement. The signal is not a restatement of output confidence: on grounded QA an out-of-fold test shows it adds error-predictive information beyond single-pass confidence and entropy, concentrated in \emph{confident-but-fragile} predictions, where acting on it roughly halves the retained error of a confidence filter. The distinctness is regime-graded, so ASMI predicts its own domain of applicability, strong where answers are routed through provided context and bounded by design where they are recalled from parametric knowledge. Sem-ASMI reads the signal from a single greedy response, without the stochastic generations the strongest baselines require, and ties or beats Semantic Entropy on ten of the twelve grounded benchmark-backbone settings. Across the same twelve settings, the best ASMI variant, typically the adaptive one reusing the ten samples already drawn for the baselines, ties or leads the strongest baseline in eight, significantly in three under a paired test. On parametric QA all variants revert to or below the zero-cost MSP baseline, exactly as predicted, and the estimates are near-deterministic across reruns. A head-level analysis shows that what tracks this boundary is not the presence of head-level fragility but whether that fragility couples to errors.

大模型不确定性注意力机制稳健性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。