arXiv:2606.14900cs.LG2026-06

GRASP实现高效多源迁移学习,内存恒定且精度超集成方法。

GRASP: Gradient-Aligned Sequential Parameter Transfer for Memory-Efficient Multi-Source Learning

论文配图:GRASP: Gradient-Aligned Sequential Parameter Transfer for Memory-Efficient Multi-Source Learning
图 1 · 摘自论文原文
  • 逐个融合源模型,参数梯度对齐选择有效知识
  • 跨10到108年数据分布的测试中平均准确率达93.5%
  • 适合资源受限场景,支持无限数量源模型扩展

多源迁移学习面临可扩展性瓶颈:现有方法要么需同时加载所有K个源模型进行参数融合(内存开销O(K)),要么在推理时部署全部模型,难以用于实际部署。本文提出GRASP(Gradient-Aligned Sequential Parameter Transfer),通过三项创新实现高性能知识融合且内存恒定(O(1)):(1)逐个顺序将源模型融合进动态目标模型;(2)参数级梯度对齐,仅转移优化方向一致的参数,避免负迁移;(3)迭代微调,使知识在融合下一源前完成适应。在三个持续学习基准(Yearbook、CLEAR-10、CLEAR-100)上测试,覆盖10至108年时间分布偏移,涵盖四种架构(1.3M至25.6M参数),GRASP在所有数据集与架构上平均准确率达93.5%,显著优于集成方法的71.7%。关键优势在于其序列融合机制不随源数量增加内存消耗,适用于资源受限环境及持续演化的源域。

原文摘要 · Abstract (English)

Multi-source transfer learning faces a fundamental scalability bottleneck: existing approaches require either loading all K source models into memory simultaneously during parameter fusion, requiring O(K) memory, or deploying all models at inference time, making production deployment infeasible. We propose GRASP (Gradient-Aligned Sequential Parameter Transfer), which achieves superior knowledge integration while maintaining O(1) memory consumption through three key innovations: (1) sequential processing that merges one source at a time into an evolving target model, (2) parameter-wise gradient alignment that selectively transfers only parameters whose optimization directions align with the target domain, avoiding negative transfer, and (3) iterative fine-tuning that adapts transferred knowledge before integrating the next source. Extensive experiments across three continual learning benchmarks (Yearbook, CLEAR-10, CLEAR-100) spanning 10 to 108-year temporal distribution shifts and four architectures (1.3M to 25.6M parameters) demonstrate that GRASP achieves 93.5% mean accuracy over all datasets and architectures compared to ensemble method's 71.7% accuracy while requiring only constant memory versus K models for standard multi-source fusion. Critically, GRASP's sequential previously merged models and scales to arbitrarily many sources without memory growth, making it uniquely suitable for resource-constrained deployment and continually evolving source domains.

多源迁移内存效率持续学习参数融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。