揭示推荐系统过度依赖最新行为的结构根源。
Residual Dominance as a Structural Account of Last-Item Reliance in Causal Self-Attention Recommenders
- 发现残差连接使模型输出偏向最近交互项。
- 减弱残差强度可降低对最新项的依赖性。
- 适合研究推荐系统可解释性与公平性的人参考。
基于Transformer的序列推荐模型在推理时往往严重依赖最近一次交互，但这种行为在表示结构中的体现尚不明确。本文结合推理时诊断与注意力模块的范数分析，发现SASRec类模型表现出高度局部化的最后项依赖。尽管自注意力机制会聚合上下文信息，但残差连接的加入显著将整体表示推向同位置贡献，称为残差主导。通过推理时调节残差强度作为可控干预，观察到结构混合与最后项依赖之间存在单调权衡；减弱残差后，部分原本被忽略的正确项在非末位位置已具备正确排序能力。结果为极端最后项依赖提供了结构层面的解释。代码已公开。
原文摘要 · Abstract (English)
Transformer-based sequential recommenders with causal self-attention often rely heavily on the most recent interaction at inference time, but how this behavior is structurally expressed in the representation used for prediction remains unclear. We combine prediction-time diagnostics with norm-based analysis of the full attention block. First, we show that SASRec-style models exhibit highly localized last-item reliance. We then find that, although self-attention aggregates contextual information, residual addition sharply shifts the full-block representation toward same-position contributions, which we term residual dominance. To probe this interpretation, we use inference-time residual scaling as a controlled diagnostic intervention. Changing the residual strength induces a monotonic trade-off between structural mixing and last-item reliance, while reducing residual strength recovers a subset of final-position misses for which representations at non-final positions already rank the ground-truth item correctly. Our results provide a structural account linking extreme last-item reliance to residual dominance at inference time. The code is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。