arXiv:2512.09910cs.CLcs.AI2025-12

用低秩适配实现高效持续翻译,避免遗忘且无需重训。

Efficient Continual Learning in Neural Machine Translation: A Low-Rank Adaptation Approach

  • 用低秩矩阵更新替代全参数微调,节省大量计算资源。
  • 新方法可实时切换语言风格,用户操控无需重新训练。
  • 基于历史梯度的正则化策略,有效防止旧知识丢失。

神经机器翻译中的持续学习面临灾难性遗忘和高计算成本两大挑战。本文将低秩适配(LoRA)作为参数高效框架,应用于专用NMT架构。实验表明,基于LoRA的微调在新语言和新领域上性能与全参数方法相当,仅使用极少参数空间。进一步提出一种交互式适配方法,通过校准的LoRA模块线性组合,实现无门控的专家混合,支持实时、用户可控的风格与领域切换,无需重训。最后,为缓解灾难性遗忘,设计了一种针对低秩分解矩阵的梯度正则化策略,利用历史梯度信息加权惩罚低秩更新,而非全参数。实验显示该方法在保留旧任务知识的同时高效学习新任务,提供可扩展的交互式持续翻译范式。

原文摘要 · Abstract (English)

Continual learning in Neural Machine Translation (NMT) faces the dual challenges of catastrophic forgetting and the high computational cost of retraining. This study establishes Low-Rank Adaptation (LoRA) as a parameter-efficient framework to address these challenges in dedicated NMT architectures. We first demonstrate that LoRA-based fine-tuning adapts NMT models to new languages and domains with performance on par with full-parameter techniques, while utilizing only a fraction of the parameter space. Second, we propose an interactive adaptation method using a calibrated linear combination of LoRA modules. This approach functions as a gate-free mixture of experts, enabling real-time, user-controllable adjustments to domain and style without retraining. Finally, to mitigate catastrophic forgetting, we introduce a novel gradient-based regularization strategy specifically designed for low-rank decomposition matrices. Unlike methods that regularize the full parameter set, our approach weights the penalty on the low-rank updates using historical gradient information. Experimental results indicate that this strategy efficiently preserves prior domain knowledge while facilitating the acquisition of new tasks, offering a scalable paradigm for interactive and continual NMT.

持续学习低秩适配机器翻译参数效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。