arXiv:2607.04270cs.IRcs.AI2026-07

解决大模型推荐中因物品描述长度差异导致的偏差问题。

LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation

论文配图:LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation
图 1 · 摘自论文原文
  • 通过长度感知注意力校准,平衡不同长度描述的注意力分配。
  • 用信息论替代简单计数,改进输出端长度归一化方法。
  • 在多个真实数据集上提升推荐准确率与公平性,开销极小。

大语言模型(LLM)通过将推荐任务转化为文本生成,展现出强大潜力。然而我们发现了一个普遍存在却未被充分研究的问题:长度偏差。由于物品描述长度不一,模型在输入侧会过度关注长描述,在输出侧则天然偏向短物品。传统长度归一化甚至可能引入新偏差并降低性能。为此,我们提出轻量级、通用的LBR框架,从输入和输出两方面缓解该问题。输入侧采用长度感知注意力校准,通过长度相关的偏置项修正注意力分布;输出侧引入有效信息长度归一化,基于前缀树分支结构计算信息理论意义上的长度代理。在三个真实世界Amazon数据集及两种主流LLM推荐模型上的实验表明,LBR显著减轻长度偏差,持续提升推荐准确率与公平性,平均NDCG@5提升16.82%,训练与推理开销几乎可忽略。

原文摘要 · Abstract (English)

Large language models (LLMs) have recently emerged as powerful backbones for recommender systems by reformulating recommendation as a token-level generation task. Despite their promise, we identify a pervasive yet underexplored issue: $\textit{Length Bias}$. Because items are represented by textual descriptions of varying lengths, LLM-based recommenders can be systematically biased in two ways. On the input side, longer item descriptions occupy more tokens in the context and thus receive disproportionately large aggregate attention mass during user preference modeling. On the output side, decoding based on summed autoregressive log-likelihood score inherently disfavors long items. Worse still, conventional length normalization can introduce an additional bias and even degrade recommendation performance. To address this problem, we propose $\textbf{LBR}$ ($\textbf{L}$ength $\textbf{B}$ias $\textbf{R}$eduction), a lightweight and model-agnostic framework for mitigating length bias in LLM-based recommendation. LBR mitigates input-side bias via Length-Aware Attention Calibration, which incorporates a length-dependent offset into attention logits to neutralize attention skew. For the output side, LBR introduces Effective Information Length Normalization, replacing naive token count with an information-theoretic length surrogate derived from the branching structure of the prefix tree. Extensive experiments on three real-world Amazon datasets and two representative LLM-based recommenders demonstrate that LBR substantially alleviates length bias while consistently improving recommendation accuracy and fairness, with negligible additional training and inference overhead (with an average NDCG@5 gain of 16.82%). The code is available at https://github.com/Void-JackLee/LBR.

推荐系统大模型长度偏差注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。