提出双架构框架，让网络同时具备记忆稳定性和学习灵活性。
Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspective
- 用两个独立轻量网络分别专注稳定与灵活学习
- 相同参数下更深网络更易学新知识，更宽网络更会保留旧知识
- 可无缝接入现有方法，参数量少87%却性能更强
持续学习旨在使神经网络具备增量式学习与适应能力。核心挑战在于解决稳定性与可塑性之间的权衡：既要保留已有知识，又要学习新知识。现有方法多聚焦于参数层面的平衡，常忽略网络架构对这一权衡的影响。本文从架构角度重新审视该矛盾，发现同等参数约束下，深层网络更具可塑性，而宽层网络则表现出更强的稳定性。为此，我们提出一种名为Dual-Arch的新框架，作为持续学习的即插即用组件。该框架利用两个独立且专用的轻量级网络：一个专攻可塑性，另一个专攻稳定性。大量实验表明，Dual-Arch能显著提升现有持续学习方法的性能，且参数量最多减少87%。代码已开源：https://github.com/byyx666/Dual-Arch。
原文摘要 · Abstract (English)
The quest for Continual Learning (CL) seeks to empower neural networks with the ability to learn and adapt incrementally. Central to this pursuit is addressing the stability-plasticity dilemma, which involves striking a balance between two conflicting objectives: preserving previously learned knowledge and acquiring new knowledge. While numerous CL methods aim to achieve this trade-off, they often overlook the impact of network architecture on stability and plasticity, restricting the trade-off to the parameter level. In this paper, we delve into the conflict between stability and plasticity at the architectural level. We reveal that under an equal parameter constraint, deeper networks exhibit better plasticity, while wider networks are characterized by superior stability. To address this architectural-level dilemma, we introduce a novel framework denoted Dual-Arch, which serves as a plug-in component for CL. This framework leverages the complementary strengths of two distinct and independent networks: one dedicated to plasticity and the other to stability. Each network is designed with a specialized and lightweight architecture, tailored to its respective objective. Extensive experiments demonstrate that Dual-Arch enhances the performance of existing CL methods while being up to 87% more compact in terms of parameters. Code: https://github.com/byyx666/Dual-Arch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。