发现开源大模型预填充攻击普遍存在，且多数模型难以防御。
Exposing the Systematic Vulnerability of Open-Weight Models to Prefill Attacks
- 利用预填充初始输出令牌，绕过模型安全机制
- 20余种策略测试中，所有主流开源模型均被攻破
- 提示特定模型的定制化攻击更有效，适合安全研究人员
随着大语言模型能力提升，其滥用风险也随之增加。封闭源代码模型通常依赖外部防护，而开源权重模型则主要依靠内部机制防范有害行为。以往红队测试多关注输入层面的越狱和参数篡改，但开源模型天然支持预填充功能，允许攻击者在生成前预先设定部分输出令牌。尽管该攻击路径潜力巨大，却未得到系统性研究。本文首次对预填充攻击开展最大规模实证研究，评估了超过20种现有及新型策略，在多个模型家族和前沿开源模型上进行测试。结果表明，预填充攻击对所有主流现代开源模型均具持续有效性，揭示了一个此前严重低估的关键漏洞，对部署具有重大影响。虽然某些大型推理模型对通用预填充具有一定鲁棒性，但仍易受针对具体模型的定制化策略攻击。研究强调，模型开发者亟需将预填充攻击防御纳入优先考虑范畴。
原文摘要 · Abstract (English)
As the capabilities of large language models continue to advance, so does their potential for misuse. While closed-source models typically rely on external defenses, open-weight models must primarily depend on internal safeguards to mitigate harmful behavior. Prior red-teaming research has largely focused on input-based jailbreaking and parameter-level manipulations. However, open-weight models also natively support prefilling, which allows an attacker to predefine initial response tokens before generation begins. Despite its potential, this attack vector has received little systematic attention. We present the largest empirical study to date of prefill attacks, evaluating over 20 existing and novel strategies across multiple model families and state-of-the-art open-weight models. Our results show that prefill attacks are consistently effective against all major contemporary open-weight models, revealing a critical and previously underexplored vulnerability with significant implications for deployment. While certain large reasoning models exhibit some robustness against generic prefilling, they remain vulnerable to tailored, model-specific strategies. Our findings underscore the urgent need for model developers to prioritize defenses against prefill attacks in open-weight LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。