通过向量对齐解决大模型安全响应的两难问题。
LLM-VA: Resolving the Jailbreak-Overrefusal Trade-off via Vector Alignment
- 用闭式权重更新让回答意愿与安全判断向量对齐
- 在12个模型上提升11.45%的F1，保留95.92%可用性
- 无需微调或改架构，自动适配不同模型的安全偏好
安全对齐的大语言模型存在两种失效模式：越狱（回应有害输入）和过度拒绝（拒绝无害查询）。现有向量操控方法调节回答向量的幅度，但导致根本性权衡——降低越狱会增加过度拒绝，反之亦然。我们发现根源在于：大模型将回答决策（回答向量 $v_a$）与输入安全性判断（良性向量 $v_b$）编码为近乎正交方向，视为独立过程。我们提出 LLM-VA，通过闭式权重更新将 $v_a$ 与 $v_b$ 对齐，使回答意愿因果依赖于安全评估——无需微调或架构修改。方法使用 SVM 在每层识别向量，筛选安全相关层，并通过最小范数权重调整迭代对齐。12个大模型的实验表明，LLM-VA 相比最佳基线提升 11.45% 的 F1，同时保持 95.92% 的模型可用性，且自动适应各模型的安全偏倚，无需人工调参。代码与模型已开源。
原文摘要 · Abstract (English)
Safety-aligned LLMs suffer from two failure modes: jailbreak (answering harmful inputs) and over-refusal (declining benign queries). Existing vector steering methods adjust the magnitude of answer vectors, but this creates a fundamental trade-off -- reducing jailbreak increases over-refusal and vice versa. We identify the root cause: LLMs encode the decision to answer (answer vector $v_a$) and the judgment of input safety (benign vector $v_b$) as nearly orthogonal directions, treating them as independent processes. We propose LLM-VA, which aligns $v_a$ with $v_b$ through closed-form weight updates, making the model's willingness to answer causally dependent on its safety assessment -- without fine-tuning or architectural changes. Our method identifies vectors at each layer using SVMs, selects safety-relevant layers, and iteratively aligns vectors via minimum-norm weight modifications. Experiments on 12 LLMs demonstrate that LLM-VA achieves 11.45% higher F1 than the best baseline while preserving 95.92% utility, and automatically adapts to each model's safety bias without manual tuning. Code and models are available at https://hotbento.github.io/LLM-VA-Web/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。