通过语义等价对抗攻击，揭示大模型内在幻觉漏洞。
Eliciting Intrinsic Hallucinations in LLMs via Semantically Equivalent Adversarial Attacks
- 用对抗优化生成语义不变的查询变体，测试模型稳定性。
- 最差情况使上下文忠实度下降50%，连GPT-5-mini也难幸免。
- 适合关注模型可信性与知识对齐的研究者参考。
大型语言模型（LLMs）常结合外部知识源以提升事实准确性并减少幻觉，如检索增强生成（RAG）。然而，这些系统仍易受内在幻觉影响，即模型生成未经检索证据支持的虚假信息。本文提出一种新框架，通过对抗优化方法生成用户查询的自然语义等价变体，对模型进行压力测试。该框架施加严格的语义等价约束和内在幻觉目标，应用于白盒、灰盒及黑盒多种对抗攻击场景。在3个数据集上评估5个开源与5个闭源生成模型，结果表明，即使最先进的模型也极易受语义保持扰动影响，导致上下文忠实度显著下降（最高达50%），说明在上下文证据使用上的可靠性依然脆弱。研究呼吁设计更鲁棒的架构与训练目标，实现对表面查询形式无关的稳定知识锚定。代码已公开于：https://github.com/atriviveksharma/intrinsic_hall
原文摘要 · Abstract (English)
Large language models (LLMs) are often used in conjunction with external knowledge sources to improve their factual accuracy and decrease hallucinations, through methods such as Retrieval-Augmented Generation (RAG). However, these systems remain susceptible to intrinsic hallucinations, where the model generates unfaithful or fabricated information that is not supported by the retrieved evidence. We propose a novel framework to assess model robustness against this phenomenon by stress-testing using natural, semantically equivalent variations of a user query found via adversarial optimization methods. We apply our framework, which enforces strict semantic equivalence constraints and an intrinsic hallucination objective, to a range of adversarial attack techniques across white-box, gray-box, and black-box adversarial settings. Evaluating these attacks on 5 open-source and 5 closed-source generator models across 3 datasets, we demonstrate that even state-of-the-art models are highly susceptible to meaning-preserving perturbations, which significantly degrade contextual faithfulness (by up to 50% for GPT-5-mini). Our findings indicate that faithful use of in-context evidence remains fragile even in state-of-the-art LLMs, motivating architectures and training objectives that enforce robust grounding independent of surface query form. Code is available at: https://github.com/atriviveksharma/intrinsic_hall
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。