arXiv:2602.06359cs.LGcs.AI2026-02被引 2

通过梯度正交性筛选数据,高效实现领域适配且不遗忘通用能力。

Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation

  • 用轻量导航模型与强化学习动态选梯度正交的样本
  • 在医疗、法律、金融域提升性能,训练效率更高
  • 无需修改优化器,避免灾难性遗忘,适合工业级部署

为专用领域微调大语言模型常面临领域专长与通用推理能力之间的权衡,即灾难性遗忘。现有方法存在两难:梯度手术类方法虽具几何安全性但需在线投影,计算成本高昂;高效数据选择方法虽降低开销,却忽略冲突梯度方向。本文提出正交梯度选择(OGS),一种以数据为中心的方法,兼顾领域表现、通用能力保留与训练效率。OGS将梯度投影的几何洞察从优化器转移到数据选择阶段,将数据选择视为带约束的决策过程。通过轻量导航模型与强化学习技术,动态识别与通用知识锚点梯度正交的训练样本。该方法确保目标模型更新自然安全,无需修改优化器或运行时投影。在医疗、法律、金融等领域的实验表明,OGS显著提升领域性能与训练效率,同时保持甚至增强在GSM8K等通用任务上的表现。

原文摘要 · Abstract (English)

Fine-tuning large language models (LLMs) for specialized domains often necessitates a trade-off between acquiring domain expertise and retaining general reasoning capabilities, a phenomenon known as catastrophic forgetting. Existing remedies face a dichotomy: gradient surgery methods offer geometric safety but incur prohibitive computational costs via online projections, while efficient data selection approaches reduce overhead but remain blind to conflict-inducing gradient directions. In this paper, we propose Orthogonal Gradient Selection (OGS), a data-centric method that harmonizes domain performance, general capability retention, and training efficiency. OGS shifts the geometric insights of gradient projection from the optimizer to the data selection stage by treating data selection as a constrained decision-making process. By leveraging a lightweight Navigator model and reinforcement learning techniques, OGS dynamically identifies training samples whose gradients are orthogonal to a general-knowledge anchor. This approach ensures naturally safe updates for target models without modifying the optimizer or incurring runtime projection costs. Experiments across medical, legal, and financial domains demonstrate that OGS achieves excellent results, significantly improving domain performance and training efficiency while maintaining or even enhancing performance on general tasks such as GSM8K.

领域适配大模型微调数据筛选

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。