用大模型生成能骗过搜索排名的恶意内容,效果远超旧方法。
Led to Mislead: Adversarial Content Injection for Attacks on Neural Ranking Models

- 基于大模型自动生成欺骗性文本,分三步优化攻击效果。
- 在多个数据集上提升排名成功率,最高达87.3%且保持语义自然。
- 适用于各类搜索引擎模型,适合研究安全或防御的开发者看。
神经排序模型(NRMs)是现代信息检索的核心,但极易受到对抗性干扰。现有攻击多依赖启发式或代理模型,效果和迁移性有限。本文提出CRAFT,一种基于大语言模型(LLM)的监督式黑盒对抗排序攻击框架。CRAFT包含三个阶段:通过检索增强生成与自我精炼构建对抗数据集,对精选对抗样本进行监督微调,以及通过偏好引导优化使生成内容更符合排名提升目标。在MS MARCO段落数据集、TREC Deep Learning 2019和2020基准上的实验表明,CRAFT显著优于当前最优基线,实现更高的排名提升率与促进率,同时保持文本流畅性和语义一致性。此外,CRAFT在跨多种排序架构(包括交叉编码器、嵌入式及基于LLM的排序器)间具有强迁移能力,揭示了真实检索系统中的安全隐患。本工作为研究神经排序模型的对抗威胁提供了系统框架,警示生成式AI在排名操控中的风险,并为构建更鲁棒的检索系统奠定基础。为保障可复现性,我们公开发布源代码、训练模型与提示模板。
原文摘要 · Abstract (English)
Neural Ranking Models (NRMs) are central to modern information retrieval but remain highly vulnerable to adversarial manipulation. Existing attacks often rely on heuristics or surrogate models, limiting effectiveness and transferability. We propose CRAFT, a supervised framework for black-box adversarial rank attacks powered by large language models (LLMs). CRAFT operates in three stages: adversarial dataset generation via retrieval-augmented generation and self-refinement, supervised fine-tuning on curated adversarial examples, and preference-guided optimization to align generations with rank-promotion objectives. Extensive experiments on the MS MARCO passage dataset, TREC Deep Learning 2019, and TREC Deep Learning 2020 benchmarks show that CRAFT significantly outperforms state-of-the-art baselines, achieving higher promotion rates and rank boosts while preserving fluency and semantic fidelity. Moreover, CRAFT transfers effectively across diverse ranking architectures, including cross-encoder, embedding-based, and LLM-based rankers, underscoring vulnerabilities in real-world retrieval systems. This work provides a principled framework for studying adversarial threats in NRMs, underscores the risks of generative AI in rank manipulation, and provides a foundation for developing more robust retrieval systems. To support reproducibility, we publicly release our source code, trained models, and prompt templates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。