针对大模型错误集中分布的特点,精准修复问题层,提升修复效率同时减少性能损失。
IRepair: An Intent-Aware Approach to Repair Data-Driven Errors in Large Language Models
- 通过动态切片识别高敏感层,仅对易错部分进行修复。
- 修复效果比基线高43.6%,性能干扰降低46%。
- 适合需要高效、低损伤修复的模型优化场景。
大型语言模型(LLM)虽表现出色,却常因数据集偏见导致毒性等错误。现有领域自适应训练方法通常全参数无差别修复,影响模型泛化能力。本文提出一种基于动态切片的意图感知修复策略IRepair,仅针对最易出错的模型层进行修复。实验在GPT2与GPT-Neo系列三个模型(800M至1.6B参数)上验证,结果表明:相比直接偏好优化(DPO)基线,IRepair在毒性缓解任务中修复效率提升43.6%,对整体性能的负面影响降低46%。分析显示,错误高度集中在少数层中,前20%层的错误密度是其余80%的7.73倍。这证明了选择性修复的必要性。此外,动态选择机制对分散错误具有更强适应性,确保修复稳健高效。
原文摘要 · Abstract (English)
Not a day goes by without hearing about the impressive feats of large language models (LLMs), and equally, not a day passes without hearing about their challenges. LLMs are notoriously vulnerable to biases in their dataset, leading to issues such as toxicity. While domain-adaptive training has been employed to mitigate these issues, these techniques often address all model parameters indiscriminately during the repair process, resulting in poor repair quality and reduced model versatility. In this paper, we introduce a novel dynamic slicing-based intent-aware LLM repair strategy, IRepair. This approach selectively targets the most error-prone sections of the model for repair. Specifically, we propose dynamically slicing the model's most sensitive layers that require immediate attention, concentrating repair efforts on those areas. This method enables more effective repairs with potentially less impact on the model's overall performance by altering a smaller portion of the model. We evaluated our technique on three models from the GPT2 and GPT-Neo families, with parameters ranging from 800M to 1.6B, in a toxicity mitigation setup. Our results show that IRepair repairs errors 43.6% more effectively while causing 46% less disruption to general performance compared to the closest baseline, direct preference optimization. Our empirical analysis also reveals that errors are more concentrated in a smaller section of the model, with the top 20% of layers exhibiting 773% more error density than the remaining 80\%. This highlights the need for selective repair. Additionally, we demonstrate that a dynamic selection approach is essential for addressing errors dispersed throughout the model, ensuring a robust and efficient repair.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。