arXiv:2604.27599cs.IRcs.LG2026-04中稿 · SIGIR 2026被引 4

让大模型推荐不因候选顺序改变而乱序,一次前向传播搞定稳定排序。

One Pass, Any Order: Position-Invariant Listwise Reranking for LLM-Based Recommendation

论文配图:One Pass, Any Order: Position-Invariant Listwise Reranking for LLM-Based Recommendation
图 1 · 摘自论文原文
  • 用结构化注意力掩码和共享位置编码,从架构上消除顺序依赖
  • 单次前向传播实现排名,相比多轮打分提升效率且结果稳定
  • 适合追求高效可靠的大模型推荐系统开发者

大型语言模型(LLM)被广泛用于推荐排序,但其列表式预测易受候选项目排列顺序影响。这导致推荐的集合性质与解码器仅依赖序列计算之间存在矛盾:相同候选集的不同排列会改变项目得分和最终排名。这种顺序敏感性使模型可靠性下降,排名可能反映提示串行而非用户偏好。本文提出 InvariRank,一种在架构层面解决该问题的排列不变列表式重排序框架。通过结构化注意力掩码阻断跨候选注意力,并在旋转位置嵌入(RoPE)下采用共享位置框架抵消位置带来的评分偏差。结合列表式学习排序目标,模型可在单次前向传播中对所有候选项进行评分,无需依赖多次排列训练。在多个推荐基准上的实验表明,InvariRank 在保持竞争力的同时,能有效应对候选顺序变化,实现稳定排名。结果表明,架构级不变性是实现可靠、高效大模型推荐重排序的可行路径。源代码见 https://github.com/ejbito/InvariRank。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly used for recommendation reranking, but their listwise predictions can depend on the order in which candidates are presented. This creates a mismatch between the set-based nature of recommendation and the sequence-based computation of decoder-only LLMs, where permuting an otherwise identical candidate set can change item scores and final rankings. Such order sensitivity makes LLM-based rerankers difficult to rely on, since rankings may reflect prompt serialization rather than user preference. We propose InvariRank, a permutation-invariant listwise reranking framework that addresses this dependence at the architectural level. InvariRank blocks cross-candidate attention with a structured attention mask and negates position-induced scoring changes through shared positional framing under Rotary Positional Embeddings (RoPE). Combined with a listwise learning-to-rank objective, the model scores all candidates in a single forward pass, avoiding permutation-based invariance training objectives that require multiple permutations of a candidate set. Experiments on recommendation benchmarks show that InvariRank maintains competitive ranking effectiveness while producing stable rankings across candidate permutations. The results suggest that architectural invariance is a practical route to reliable and efficient LLM-based recommendation reranking. The source code is at https://github.com/ejbito/InvariRank.

推荐系统大模型排序不变性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。