用可组合的控制标记实现灵活安全对齐，避免模型过度拒绝
MOSAIC: Composable Safety Alignment with Modular Control Tokens
- 通过可学习的控制标记实现模块化安全规则
- 在保持模型能力的同时，过拒率显著降低
- 适合需要动态安全策略的多场景应用
大型语言模型的安全对齐通常以嵌入模型参数的单一静态策略实现。然而，实际部署常需随上下文变化的安全规则，覆盖不同用户、地区和应用场景。现有方法难以提供此类条件控制：参数级对齐将安全行为与通用能力耦合，而基于提示的方法依赖自然语言指令，执行力度弱。本文提出MOSAIC，一种基于冻结主干模型的可组合安全对齐框架，通过可学习的控制标记实现。每个标记代表一个安全约束，可在推理时灵活激活与组合。为高效训练可组合标记，引入基于顺序的任务采样和分布级对齐目标，缓解过拒问题。实验表明，MOSAIC在保持模型效用的同时，实现更强的防御性能，过拒率显著降低。
原文摘要 · Abstract (English)
Safety alignment in large language models (LLMs) is commonly implemented as a single static policy embedded in model parameters. However, real-world deployments often require context-dependent safety rules that vary across users, regions, and applications. Existing approaches struggle to provide such conditional control: parameter-level alignment entangles safety behaviors with general capabilities, while prompt-based methods rely on natural language instructions that provide weak enforcement. We propose MOSAIC, a modular framework that enables compositional safety alignment through learnable control tokens optimized over a frozen backbone model. Each token represents a safety constraint and can be flexibly activated and composed at inference time. To train compositional tokens efficiently, we introduce order-based task sampling and a distribution-level alignment objective that mitigates over-refusal. Experiments show that MOSAIC achieves strong defense performance with substantially lower over-refusal while preserving model utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。