LLM文本排序存在决策盲区，可被恶意内容方操控排名。
The Ranking Blind Spot: Decision Hijacking in LLM-based Text Ranking
- 通过指令劫持与标准劫持，操控LLM的排序判断逻辑。
- 实验验证攻击在多种LLM和排序方案中有效，强模型更易受攻。
- 适合关注大模型安全、信息推荐公平性的研究者阅读。
大型语言模型（LLMs）在段落排序等信息检索任务中表现优异。本研究探讨了LLM的指令遵循能力如何影响多文档比较任务，识别出一种称为“排序盲区”的特征，即LLM在对比评估中的决策机制缺陷。我们通过两种方法分析该盲区的影响：决策目标劫持，改变成对排序系统的目标；决策标准劫持，修改不同排序方案中的相关性标准。这些攻击可使内容提供者操纵基于LLM的排序系统，强制特定段落获得最高排名。实验表明，这些攻击在多种LLM上均有效，且可泛化至多个排序方案。我们还发现更强的LLM反而更易受攻击。代码已开源：https://github.com/blindspotorg/RankingBlindSpot。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated strong performance in information retrieval tasks like passage ranking. Our research examines how instruction-following capabilities in LLMs interact with multi-document comparison tasks, identifying what we term the "Ranking Blind Spot", a characteristic of LLM decision processes during comparative evaluation. We analyze how this ranking blind spot affects LLM evaluation systems through two approaches: Decision Objective Hijacking, which alters the evaluation goal in pairwise ranking systems, and Decision Criteria Hijacking, which modifies relevance standards across ranking schemes. These approaches demonstrate how content providers could potentially influence LLM-based ranking systems to affect document positioning. These attacks aim to force the LLM ranker to prefer a specific passage and rank it at the top. Malicious content providers can exploit this weakness, which helps them gain additional exposure by attacking the ranker. In our experiment, We empirically show that the proposed attacks are effective in various LLMs and can be generalized to multiple ranking schemes. We apply these attack to realistic examples to show their effectiveness. We also found stronger LLMs are more vulnerable to these attacks. Our code is available at: https://github.com/blindspotorg/RankingBlindSpot
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。