让大模型生成时自动纠错,既安全又不伤体验。
CARE: Decoding Time Safety Alignment via Rollback and Introspection Intervention
- 用守卫模型实时检测不安全内容,提前干预。
- 通过回滚机制快速修正错误输出,减少用户感知干扰。
- 让模型自我反思并修正前文,提升自纠正能力。
随着大语言模型在真实场景中广泛应用,其生成过程中的安全性成为关键挑战。现有解码阶段的干预方法(如对比解码)常导致安全与响应质量之间的严重权衡。本文提出CARE框架,集成三大组件:(1) 守卫模型实现实时安全监控,检测潜在不安全内容;(2) 带有标记缓冲区的回滚机制,在早期阶段高效纠正不安全输出,避免影响用户体验;(3) 创新的基于自我反思的干预策略,模型生成对自身输出的自省式批判,并将这些反思纳入上下文以指导后续解码。该框架通过守卫模型实现精准干预、回滚机制实现及时修正、以及自省方法实现有效自我修正,显著改善了安全与质量的平衡。实验表明,该框架在保持高响应质量的同时,实现了低有害响应率且对用户干扰极小。
原文摘要 · Abstract (English)
As large language models (LLMs) are increasingly deployed in real-world applications, ensuring the safety of their outputs during decoding has become a critical challenge. However, existing decoding-time interventions, such as Contrastive Decoding, often force a severe trade-off between safety and response quality. In this work, we propose CARE, a novel framework for decoding-time safety alignment that integrates three key components: (1) a guard model for real-time safety monitoring, enabling detection of potentially unsafe content; (2) a rollback mechanism with a token buffer to correct unsafe outputs efficiently at an earlier stage without disrupting the user experience; and (3) a novel introspection-based intervention strategy, where the model generates self-reflective critiques of its previous outputs and incorporates these reflections into the context to guide subsequent decoding steps. The framework achieves a superior safety-quality trade-off by using its guard model for precise interventions, its rollback mechanism for timely corrections, and our novel introspection method for effective self-correction. Experimental results demonstrate that our framework achieves a superior balance of safety, quality, and efficiency, attaining a low harmful response rate and minimal disruption to the user experience while maintaining high response quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。