arXiv:2609.01807cs.LGcs.AI2026-09

用匈牙利算法一次解出排序结果,让大模型生成式排序快64倍

hLLM: Single Pass Decoding for Generative Reranking

  • 将排序输出转为二分图匹配问题,用匈牙利算法一次性求解
  • 在N个物品排序任务中仅需1次前向传播,推理速度提升64倍
  • 适合需要实时排序的场景,如推荐系统和搜索排序

大语言模型在生成式排序上表现优异,但传统自回归解码需逐词生成,效率低下。本文发现排序只需输出N个序号(即物品排名顺序),其排列结构可被高效建模。提出hLLM(匈牙利语言模型)——一种专用解码策略:从LLM预填充隐藏状态读取N×K的项目-位置得分矩阵,通过轻量级自注意力头提取后,使用匈牙利算法一次性求解最优二分匹配,直接得到有效排列。结合LoRA微调与教师排序蒸馏,实现28毫秒端到端推理,较原方法提速64倍,且排序质量与教师模型相当。通过完整消融实验分解了架构、训练信号与主干适配的贡献。该框架将生成式排序与组合优化连接,为实时排序提供新路径。

原文摘要 · Abstract (English)

Large language models (LLMs) achieve state-of-the-art generative ranking quality, but the ranking they produce must be decoded, and autoregressive decoding spends one sequential forward pass per emitted token. We observe that the only tokens a ranker must emit are the $N$ ordinal values naming the items in ranked order, and that this narrow, permutation-structured output format admits decoding strategies which are much more efficient than left-to-right generation. We introduce hLLM (Hungarian LLM), a format-specialized decoding strategy that decodes all $N$ ordinals in $O(1)$ forward passes. hLLM reads an $N \times K$ item-position score matrix off the LLM's prefill hidden states with a lightweight self-attention head, then decodes the ordinals as the optimal bipartite assignment of that matrix via the Hungarian algorithm, yielding a valid permutation by construction rather than by repair. Through a systematic study of training signals and backbone adaptation, we show that LoRA-based fine-tuning combined with teacher ranking distillation reaches 28 ms end-to-end inference, a speed-up of $64\times$ while maintaining ranking quality on par with the teacher. We provide a complete ablation decomposing the contributions of architecture, training signal, and backbone adaptation. Our framework connects generative ranking to combinatorial optimization, opening a path toward other $O(1)$-decode mechanisms for real-time ranking.

排序优化推理加速匈牙利算法大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。