动态剪枝时保留关键安全电路，提升大模型推理安全性。
Alignment-Constrained Dynamic Pruning for LLMs: Identifying and Preserving Alignment-Critical Circuits
- 基于探测剪枝思想，动态识别并保护与对齐相关的关键计算路径。
- 在相同计算量下，拒绝率提升50%，显著增强模型安全性。
- 适合需要高效且安全部署大语言模型的场景。
大语言模型推理需大量计算资源，带来部署挑战。动态剪枝通过自适应选择计算路径，相比静态方法效率更高，但会因仅保留输入相关的安全关键路径而加剧对齐退化问题。为此，我们提出对齐感知探测剪枝（AAPP），一种动态结构化剪枝方法，在推理过程中自适应保留与对齐相关的电路。在LLaMA 2-7B、Qwen2.5-14B-Instruct和Gemma-3-12B-IT上的实验表明，AAPP在匹配计算量下使拒绝率提升50%，实现了高效且安全的大模型部署。
原文摘要 · Abstract (English)
Large Language Models require substantial computational resources for inference, posing deployment challenges. While dynamic pruning offers superior efficiency over static methods through adaptive circuit selection, it exacerbates alignment degradation by retaining only input-dependent safety-critical circuit preservation across diverse inputs. As a result, addressing these heightened alignment vulnerabilities remains critical. We introduce Alignment-Aware Probe Pruning (AAPP), a dynamic structured pruning method that adaptively preserves alignment-relevant circuits during inference, building upon Probe Pruning. Experiments on LLaMA 2-7B, Qwen2.5-14B-Instruct, and Gemma-3-12B-IT show AAPP improves refusal rates by 50\% at matched compute, enabling efficient yet safety-preserving LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。