arXiv:2607.21951cs.IRcs.CR2026-07

通过内容篡改让指定网页在LLM推荐中冲到第1位

SIREN (Luring LLMs onto the Rocks): PAIR-Driven Preference Manipulation in Web-RAG Recommenders

  • 用23种内容污染手法迭代修改网页,固定其他检索结果
  • 在8个场景下62次成功让目标网页排名第一,平均成功率80.5%
  • 发现陈述式注入比指令式更有效,适合安全测试人员参考

本文研究网络增强型大语言模型(LLM)生成推荐结果的对抗性操纵。当LLM通过实时抓取网页回答推荐请求时,每篇被检索的页面都可能成为攻击入口。现有研究关注虚假商品、检索污染和排名提升,但未比较在保持检索源不变的情况下,对已获取网页的不同修改如何影响最终排名。为此,我们提出SIREN,一种基于PAIR框架的自动化攻击-评估方法,旨在将指定实体推至LLM推荐首位。SIREN使用Anthropic的网络工具获取并捕获网页,随后利用23种可解释的内容污染技术迭代编辑。自定义RAG重播平台保持源文档顺序一致,确保排名变化仅源于内容修改。在两个生产级Claude模型上,SIREN在124次技术试验中实现62次排名第1,新鲜会话中平均成功率0.805。结果显示,陈述式排名声明和种子列表普遍优于指令式注入,但效果受目标模型影响。据我们所知,这是首个在固定源上下文条件下控制性研究生产级LLM竞争性排名操纵的工作。

原文摘要 · Abstract (English)

This paper investigates the adversarial manipulation of the ranked recommendations produced by web-augmented large language models (LLMs). When an LLM answers a recommendation query by retrieving and reading live webpages, it acts as a recommender, and each retrieved page becomes a potential attack surface. Prior work has examined fabricated products, retrieval poisoning, and rank promotion. However, these studies do not compare how different edits to an already retrieved page change the model's final ranking while the surrounding source set remains unchanged. To address this gap, we propose SIREN, an automated attacker--judge method that adapts the PAIR jailbreaking loop to competitive rank manipulation, with the goal of moving a chosen entity to rank~1 in an LLM-generated recommendation. SIREN retrieves and captures webpages using Anthropic's web tools, then iteratively edits a retrieved source using an interpretable taxonomy of 23 content-poisoning techniques. The custom-RAG replay platform keeps the same sources in the same order, so changes in the model's ranking can be linked to changes in the supplied content rather than to differences in retrieval. Across two production Claude models, SIREN reaches rank~1 in 62 of 124 technique trials nested within eight query--model contexts. The payloads that reached rank~1 were then tested in fresh sessions, where they reproduced the result with a mean success rate of 0.805. Across the evaluated settings, declarative ranking claims and seeded lists were generally more effective than directive-form injections, although the strength of this difference depended on the target model. To the best of our knowledge, this is among the first controlled studies of competitive rank manipulation in production LLMs where the supplied source context is kept fixed.

LLM安全推荐系统对抗攻击

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。