通过可解释性分析压缩模型的拒绝行为，提升安全性和可信度。
Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretability
- 从残差流中识别拒绝行为的单一方向机制
- 压缩后模型的拒绝能力下降，影响可信度
- 提出轻量级方法恢复安全性能，不损失效率
大语言模型的快速发展推动了模型压缩研究，以提升其可访问性与实用性。尽管已有大量工作从安全角度探索压缩技术，但研究表明，对齐安全性的模型在压缩后常丧失可信性。与此同时，机制可解释性领域取得进展，例如发现残差流中的单一方向可跨多种模型架构调控拒绝行为。本文从可解释性驱动视角出发，研究压缩模型的安全性，深入分析拒绝机制。基于分析结果，提出一种轻量、计算高效的改进方法，可在不损害模型性能或实用性的前提下增强压缩模型的安全性。
原文摘要 · Abstract (English)
The rapid growth of large language models has spurred significant interest in model compression as a means to enhance their accessibility and practicality. While extensive research has explored model compression through the lens of safety, findings suggest that safety-aligned models often lose elements of trustworthiness post-compression. Simultaneously, the field of mechanistic interpretability has gained traction, with notable discoveries, such as the identification of a single direction in the residual stream mediating refusal behaviors across diverse model architectures. In this work, we investigate the safety of compressed models by examining the mechanisms of refusal, adopting a novel interpretability-driven perspective to evaluate model safety. Furthermore, leveraging insights from our interpretability analysis, we propose a lightweight, computationally efficient method to enhance the safety of compressed models without compromising their performance or utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。