让不同模型家族的AI安全对齐，推理时无须重训练
ALIGNBEAM : Inference-Time Alignment Transfer via Cross-Vocabulary Logit Mixing
- 通过逐词翻译锚模型日志，实现跨词汇表安全对齐
- 在对抗性测试中显著提升拒绝率，任务准确率几乎不变
- 无需修改权重，适合部署时灵活调节安全与性能
领域微调会降低大语言模型的安全性：经过微调的专业模型容易响应以领域语言构造的有害提示。现有推理时防御方法需两模型共享词汇表，无法应用于跨模型家族场景，而此场景下安全性下降最严重。本文提出ALIGNBEAM，一种无需训练的方法，在每个解码步骤中将安全锚模型的日志逐词转换至目标模型的词汇表，并由小型LLM裁判从K个候选延续中选出最安全的输出。不修改任何权重，且可在部署时调整安全-效率权衡而无需重新训练。在跨词汇表与同词汇表评估对上，ALIGNBEAM均显著提升对抗性基准上的拒绝率，同时保持任务准确率和推理开销在实用范围内。结果表明，可在推理阶段实现跨模型家族的安全对齐，且不触碰任一模型权重。
原文摘要 · Abstract (English)
Domain fine-tuning degrades the safety of large language models: fine-tuned specialists readily comply with harmful prompts framed in domain language. Existing inference-time defenses that mix logits from a safe anchor model require both models to share a vocabulary, which rules them out for the cross-family specialists where safety is most degraded. We present ALIGNBEAM, a training-free method that lifts this restriction by translating anchor logits into the target model's vocabulary token-by-token at each decoding step; a small LLM judge then selects the safest among K candidate continuations. No weights are changed, and the safety-utility trade-off can be tuned at deployment without retraining. Across both cross-vocabulary and same-vocabulary evaluation pairs, ALIGNBEAM substantially raises refusal on adversarial benchmarks while keeping task accuracy and inference overhead within practical bounds. The results show that safety alignment can be transferred between model families at inference time, without touching either model's weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。