让大模型持续学习不遗忘,用专家分工+对抗机制提升工业级自进化能力。
Self-Evolving LLMs via Continual Instruction Tuning
- 分任务专用专家+共享专家,参数独立防止遗忘。
- 引入对抗判别器过滤噪声,共享路径只传任务相关知识。
- 在腾讯视频内容审核中降本15.3%,适合需长期迭代的工业场景。
在真实工业场景中,大语言模型需持续学习以应对不断变化的任务需求,实现自我演进以适应动态数据分布。然而,现有持续学习方法如回放和参数隔离常导致灾难性遗忘:新任务训练会因过拟合新分布而损害旧任务性能,削弱泛化能力。本文提出MoE-CL,一种面向工业规模、自演化指令微调的参数高效对抗性专家混合框架。该框架采用双专家设计:(1) 每个任务配备专用LoRA专家,通过参数独立性保留任务特定知识,缓解遗忘;(2) 共享LoRA专家实现跨任务迁移。为防止共享路径传递无关噪声,引入任务感知判别器构成GAN结构,强制共享专家仅传递与任务对齐的信息。通过对抗学习,共享专家获得类判别器的通用表征,而专用专家保持任务细节,平衡知识保留与跨任务泛化,支持自演化。在公开的MTL5基准与工业级Tencent3基准上的大量实验验证了其有效性。在腾讯视频平台的内容合规审查真实A/B测试中,MoE-CL将人工审核成本降低15.3%。结果表明,MoE-CL适用于持续适应与稳定迁移至关重要的大规模工业部署场景。
原文摘要 · Abstract (English)
In real-world industrial settings, large language models (LLMs) must learn continually to keep pace with diverse and evolving tasks, requiring self-evolution to refine knowledge under dynamic data distributions. However, existing continual learning (CL) approaches, such as replay and parameter isolation, often suffer from catastrophic forgetting: training on new tasks degrades performance on earlier ones by overfitting to the new distribution and weakening generalization.We propose MoE-CL, a parameter-efficient adversarial mixture-of-experts framework for industrial-scale, self-evolving continual instruction tuning of LLMs. MoE-CL uses a dual-expert design: (1) a dedicated LoRA expert per task to preserve task-specific knowledge via parameter independence, mitigating forgetting; and (2) a shared LoRA expert to enable cross-task transfer. To prevent transferring task-irrelevant noise through the shared pathway, we integrate a task-aware discriminator within a GAN. The discriminator encourages the shared expert to pass only task-aligned information during sequential training. Through adversarial learning, the shared expert acquires generalized representations that mimic the discriminator, while dedicated experts retain task-specific details, balancing knowledge retention and cross-task generalization and thereby supporting self-evolution.Extensive experiments on the public MTL5 benchmark and an industrial Tencent3 benchmark validate the effectiveness of MoE-CL for continual instruction tuning. In real-world A/B testing for content compliance review on the Tencent Video platform, MoE-CL reduced manual review costs by 15.3%. These results demonstrate that MoE-CL is practical for large-scale industrial deployment where continual adaptation and stable transfer are critical.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。