让注意力真正指向有效证据,提升模型预测可靠性
Relevance Is Not Permission: Warranted Attention for Value Contributions

- 引入权限机制g_ij,判断注意力值是否应成为预测依据
- 在32组对比中,27次提升主指标,10次显著改善
- 适合需要精准证据选择的下游任务,如链接预测与排序
注意力虽能聚焦相关键值项,但不保证其贡献可作为预测证据。一篇相关段落未必支持答案,历史事实或时间邻近项甚至可能干扰真实尾部排名或当前边得分。本文将此差距形式化为加权值项α_ij * v_j的权限问题,提出Warrant:一种路径局部化的接口,在保留注意力相关性α_ij的基础上,显式暴露通往核心指标的值路径,并通过学习得到的查询-项权限g_ij,将α_ij * v_j转化为α_ij * g_ij * v_j。该机制应用于CTDG链接预测、MTPP下一标记排序、RAG支持证据选择、STPP下一起点预测和TKG尾部预测。在32组配对比较、3个种子、共192次运行中,Warrant在27次中提升主指标;其中10次为显著提升,1次边际提升,8次正向但不确定,8次持平/微弱影响,5次下降。路径定位验证显示,正确路径放置优于方向感知基线,且在CTDG上比通用注意力提升+0.1076 AUC,TKG上提升+0.0683 MRR。消融实验表明,TKG性能提升主要源于历史尾部值路径暴露,而核心CTDG提升来自边条件查询-项权限。结论:预测证据并非注意力质量,只有路径上的权重值被授权时才构成证据。
原文摘要 · Abstract (English)
Relevance is not permission. Attention lets a model read key-value items related to the current query, but it does not guarantee that the value contribution of such an item becomes prediction evidence. A retrieved passage may be relevant to a question without being supporting evidence, and a historical fact or temporal neighbor may even blur true-tail ranking or the current edge score. This paper formalizes this gap as a permission problem for the weighted value term alpha_ij * v_j that is actually added to the prediction path. We propose Warrant, a path-localized interface that preserves attention relevance alpha_ij, exposes the value path leading to the primary metric, and, in the full model, turns alpha_ij * v_j into alpha_ij * g_ij * v_j through learned query-item permission g_ij. We place the same operator on the metric-defining value paths of CTDG link prediction, MTPP next-mark ranking, RAG supporting evidence selection, STPP next-location forecasting, and TKG tail prediction. Across 32 paired comparisons, 3 seeds, and 192 total runs, Warrant improves the primary metric in 27 comparisons; practical tiers consist of 10 substantial effects, 1 marginal effect, 8 positive but uncertain effects, 8 tie/negligible effects, and 5 drops. In the path-localization check, correct-path placement outperforms direction-aware Base performance in every domain and exceeds generic attention placement by +0.1076 AUC in CTDG and +0.0683 MRR in TKG. Ablations show that most TKG gains come from historical-tail value path exposure, whereas the core CTDG gain comes from edge-conditioned query-item permission. In conclusion, prediction evidence is not attention mass. A weighted value term becomes evidence only when it is warranted on the path to the metric.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。