让微调后的大模型重拾诚实：仅用少量数据修复其表达认知边界的能力
Fine-Tuned LLMs Know They Don't Know: A Parameter-Efficient Approach to Recovering Honesty
- 通过识别关键神经元，将其恢复到预训练状态并补偿任务相关参数
- 在4个问答任务中恢复33.25%的诚实度，效率提升2.23倍以上
- 适合追求可信部署的开发者，尤其关注低资源场景下的模型可靠性
大型语言模型（LLMs）的诚实性对高风险场景的安全部署至关重要。然而，常见的监督微调（SFT）会严重削弱这一特质。现有方法依赖大量数据全局调整参数，隐含假设是SFT彻底破坏了模型识别知识边界的能力。我们观察到，微调后的模型仍保留该能力，受损的是其诚实地表达认知边界的能力。为此，提出诚实关键神经元恢复（HCNR），通过识别并恢复关键表达神经元至预训练状态，同时利用海森矩阵引导补偿，使其与任务导向神经元协同。在四个问答任务和五个LLM家族上的实验表明，HCNR有效恢复33.25%被削弱的诚实性，且速度比基线快至少2.23倍，所需数据量减少10倍以上，为可信LLM部署提供了实用方案。
原文摘要 · Abstract (English)
The honesty of Large Language Models (LLMs) is increasingly important for safe deployment in high-stakes domains. However, this crucial trait is severely undermined by supervised fine-tuning (SFT), a common technique for model specialization. Existing recovery methods rely on data-intensive global parameter adjustments, implicitly assuming that SFT deeply corrupts the models' ability to recognize their knowledge boundaries. However, we observe that fine-tuned LLMs still preserve this ability; what is damaged is their capacity to faithfully express that awareness. Building on this, we propose Honesty-Critical Neurons Restoration (HCNR) to surgically repair this suppressed capacity. HCNR identifies and restores key expression-governing neurons to their pre-trained state while harmonizing them with task-oriented neurons via Hessian-guided compensation. Experiments on four QA tasks and five LLM families demonstrate that HCNR effectively recovers 33.25% of the compromised honesty while achieving at least 2.23x speedup with over 10x less data compared to baseline methods, offering a practical solution for trustworthy LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。