通过检测特征空间中的恶意信号，动态调整模型拒绝机制。
Feature-Aware Malicious Output Detection and Mitigation
- 在解码阶段用判别器识别潜在恶意特征
- 检测到有害特征后重生成当前词元，成功率提升至90%以上
- 不破坏正常生成能力，适合部署于安全敏感场景
大型语言模型（LLMs）的快速发展在带来诸多便利的同时也引入了显著风险。尽管经过强化学习微调，现有模型仍无法有效识别恶意内容，难以抵御越狱攻击。为此，我们提出一种特征感知的有害响应拒绝方法（FMM），通过分析模型特征空间中的恶意特征，自适应调整拒绝机制。该方法在解码阶段引入简单判别器，检测潜在恶意特征；一旦发现毒害性标记的特征，便触发当前词元的重生成。同时，通过激活修补技术，在后续词元生成中加入额外拒绝向量，引导模型生成拒绝响应。实验表明，该方法在多种语言模型和多样攻击手段下均表现有效，且关键保持了模型的标准生成能力。
原文摘要 · Abstract (English)
The rapid advancement of large language models (LLMs) has brought significant benefits to various domains while introducing substantial risks. Despite being fine-tuned through reinforcement learning, LLMs lack the capability to discern malicious content, limiting their defense against jailbreak. To address these safety concerns, we propose a feature-aware method for harmful response rejection (FMM), which detects the presence of malicious features within the model's feature space and adaptively adjusts the model's rejection mechanism. By employing a simple discriminator, we detect potential malicious traits during the decoding phase. Upon detecting features indicative of toxic tokens, FMM regenerates the current token. By employing activation patching, an additional rejection vector is incorporated during the subsequent token generation, steering the model towards a refusal response. Experimental results demonstrate the effectiveness of our approach across multiple language models and diverse attack techniques, while crucially maintaining the models' standard generation capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。