恶意指令藏在上传文本中，可悄悄操控大模型输出。
Prompt-in-Content Attacks: Exploiting Uploaded Inputs to Hijack LLM Behavior
- 将攻击指令嵌入看似正常的用户输入内容中。
- 无需系统漏洞即可让模型生成偏见或虚假内容。
- 适合关注大模型安全的开发者与研究者阅读。
大型语言模型广泛应用于接受用户提交内容（如上传文档或粘贴文本）的场景，用于摘要和问答等任务。本文揭示了一类新型攻击——内容内提示注入，即在看似无害的输入中嵌入恶意指令。当这些隐藏提示被大模型处理时，可悄然操纵输出结果，且用户无感知、系统无入侵，导致摘要偏见、虚构事实或误导性建议。我们在多个主流平台验证了此类攻击的可行性，分析其根源在于提示拼接和输入隔离不足，并提出缓解策略。研究揭示了真实大模型工作流中一种隐蔽而实际的安全威胁。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are widely deployed in applications that accept user-submitted content, such as uploaded documents or pasted text, for tasks like summarization and question answering. In this paper, we identify a new class of attacks, prompt in content injection, where adversarial instructions are embedded in seemingly benign inputs. When processed by the LLM, these hidden prompts can manipulate outputs without user awareness or system compromise, leading to biased summaries, fabricated claims, or misleading suggestions. We demonstrate the feasibility of such attacks across popular platforms, analyze their root causes including prompt concatenation and insufficient input isolation, and discuss mitigation strategies. Our findings reveal a subtle yet practical threat in real-world LLM workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。