arXiv:2506.17848cs.LGcs.AI2025-06

PaPI通过路径路由机制,实现低能耗持续学习,避免遗忘且理论可保证。

Pathway-based Progressive Inference (PaPI) for Energy-Efficient Continual Learning

  • 基于路径的渐进推理框架,动态选择最优参数路径。
  • 理论证明其遗忘率更低,能耗随活跃参数数增长而非模型总大小。
  • 适合边缘设备等资源受限场景下的持续学习应用。

持续学习系统面临防止灾难性遗忘与保持能效的双重挑战,尤其在资源受限环境中。本文提出路径式渐进推理(PaPI),一种基于数学严谨性的新理论框架,通过路径选择与自适应机制应对该挑战。我们将持续学习建模为能量约束优化问题,并为路径路由机制提供正式收敛保证。理论分析表明,相较于单体架构,PaPI在稳定性-可塑性权衡上实现$/mathcal{O}(K)$的改进,其中$K$为路径数量。我们利用Fisher信息矩阵推导出遗忘率的紧界,并证明PaPI的能量消耗仅随活跃参数数量增长,而非模型总规模。对比理论分析显示,PaPI在防止灾难性遗忘方面优于EWC,同时能效高于EWC与GEM。实验验证了这些理论优势,在多个基准测试中均表现出色,证实了其在资源受限环境下的持续学习有效性。代码已开源:https://github.com/zser092/PAPI_FILES。

原文摘要 · Abstract (English)

Continual learning systems face the dual challenge of preventing catastrophic forgetting while maintaining energy efficiency, particularly in resource-constrained environments. This paper introduces Pathway-based Progressive Inference (PaPI), a novel theoretical framework that addresses these challenges through a mathematically rigorous approach to pathway selection and adaptation. We formulate continual learning as an energy-constrained optimization problem and provide formal convergence guarantees for our pathway routing mechanisms. Our theoretical analysis demonstrates that PaPI achieves an $\mathcal{O}(K)$ improvement in the stability-plasticity trade-off compared to monolithic architectures, where $K$ is the number of pathways. We derive tight bounds on forgetting rates using Fisher Information Matrix analysis and prove that PaPI's energy consumption scales with the number of active parameters rather than the total model size. Comparative theoretical analysis shows that PaPI provides stronger guarantees against catastrophic forgetting than Elastic Weight Consolidation (EWC) while maintaining better energy efficiency than both EWC and Gradient Episodic Memory (GEM). Our experimental validation confirms these theoretical advantages across multiple benchmarks, demonstrating PaPI's effectiveness for continual learning in energy-constrained settings. Our codes are available at https://github.com/zser092/PAPI_FILES.

持续学习能效优化路径路由

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。