arXiv:2607.18622cs.CRcs.AI2026-07中稿 · COLM

发现文本协同优化中恶意指令会通过聚合传播,导致模型性能下降。

CPInj: Uncovering Prompt Injection Risks in Textual Collaborative Prompt Optimization

论文配图:CPInj: Uncovering Prompt Injection Risks in Textual Collaborative Prompt Optimization
图 1 · 摘自论文原文
  • 设计攻击方法CPInj,让恶意指令在多方协作中持续存在并影响全局提示。
  • 在数学、逻辑、医学任务中,攻击使模型性能显著下降,且现有防御无效。
  • 提出锚定净化聚合机制,可清除恶意指令而不严重损害正常优化效果。

文本协同提示优化(TCPO)将TextGrad扩展至去中心化场景,允许多个客户端在本地数据基础上协同优化大语言模型(LLM)的提示词,同时保持数据私密。其依赖自由文本更新与聚合,引入了新的攻击面:恶意指令可被注入本地提示,并通过服务器端提示聚合传播。不同于传统提示注入攻击,本研究针对的是TCPO中的协作优化循环。此类攻击更具挑战性,因恶意指令需在聚合中存活、在后续良性优化中持续存在,并避开服务器端检测。为此,我们提出协同提示注入(CPInj)攻击,能污染全局提示,降低下游任务表现,抵抗良性客户端的净化优化,且可绕过先进检测防御。实验覆盖三个LLM家族和五个推理任务(数学、逻辑、医学),结果表明该攻击揭示了TCPO的关键漏洞。尽管提出锚定净化聚合(APAgg)作为防御方案,但攻击仍高度有效,亟需更鲁棒的防御机制。

原文摘要 · Abstract (English)

Textual Collaborative Prompt Optimization (TCPO) extends TextGrad (Yuksekgonul et al., 2025) to a decentralized setting by allowing multiple clients to jointly improve prompts for large language models (LLMs) while keeping their data locally. Its reliance on free-form textual updating and aggregation introduces a new and largely unexplored attack surface, i.e., malicious instructions can be injected into local prompts and propagated through server-side prompt aggregation. Unlike conventional prompt injection attacks, attacking TCPO targets the collaborative optimization loop in TCPO. This setting is more challenging because malicious instructions must survive aggregation, persist through subsequent benign prompt optimization, and evade server-side defenses. To expose this risk, we propose Collaborative Prompt Injection (CPInj) attack that contaminates the aggregated global prompt with malicious instructions, degrades downstream task performance, resists purification by prompt optimization on benign clients, and evades advanced detection-based defenses on the server. We find that current defense methods are ineffective against CPInj. We further propose Anchored Purification Aggregation (APAgg), a defense-oriented aggregation that purifies malicious instructions without severely degrading TCPO utility. We conduct extensive experiments across three LLM families and five reasoning tasks in math, logic, and medicine, and demonstrate that our proposed attack reveals a critical vulnerability in TCPO. Although we take a first step toward mitigation, the attack remains highly effective and far from fully resolved, calling for more robust defense for TCPO.

提示攻击协同优化大模型安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。