用错误案例迭代训练，提升专业模型性能且不损通用能力
APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Training
- 聚焦模型出错的弱例和相似样本进行针对性微调
- 在LLaMA-2和Mistral-V0.3上保持通用能力，下游任务表现更优
- 适合需要强专业性又不愿牺牲通用性的场景
大型语言模型（LLMs）常需领域微调以应对特定任务，但可能损害其通用能力。本文提出APT（弱例获取与迭代偏好训练）方法，通过自生成的低偏好弱例（错误案例及相似案例）进行训练。该方法仅使用模型出错的样本，并辅以少量检索到的相似样本，实现精准优化。实验表明，在LLaMA-2和Mistral-V0.3多个基准测试中，APT有效维持通用能力，同时显著提升下游任务性能，优于现有多种方法。验证了该策略在不牺牲通用性前提下增强领域能力的有效性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often require domain-specific fine-tuning to address targeted tasks, which risks degrading their general capabilities. Maintaining a balance between domain-specific enhancements and general model utility is a key challenge. This paper proposes a novel approach named APT (Weakness Case Acquisition and Iterative Preference Training) to enhance domain-specific performance with self-generated dis-preferred weakness data (bad cases and similar cases). APT uniquely focuses on training the model using only those samples where errors occur, alongside a small, similar set of samples retrieved for this purpose. This targeted training minimizes interference with the model's existing knowledge base, effectively retaining generic capabilities. Experimental results on the LLama-2 and Mistral-V0.3 models across various benchmarks demonstrate that APT ensures no reduction in generic capacity and achieves superior performance on downstream tasks compared to various existing methods. This validates our method as an effective strategy for enhancing domain-specific capabilities without sacrificing the model's broader applicability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。