受果蝇记忆系统启发,实现高效持续学习
FlyPrompt: Brain-Inspired Random-Expanded Routing with Temporal-Ensemble Experts for General Continual Learning
- 用随机扩展路由机制动态分配专家参数
- 在多个数据集上性能超越现有方法7.6%~12.4%
- 适合资源受限的实时持续学习场景
通用持续学习(GCL)要求智能系统从无明确任务边界的单次遍历数据流中学习。尽管预训练模型的参数高效微调(PET)有前景,但通常依赖多轮训练和显式任务提示,限制了其在GCL中的应用。现有方法缺乏针对性设计,未能解决两个核心问题:如何将专家参数分配给不断变化的数据分布,以及在有限监督下如何提升其表征能力。受果蝇层级记忆系统中稀疏扩展与模块化集成的启发,我们提出FlyPrompt,一个脑启发框架,将GCL分解为专家路由与专家能力提升两个子问题。FlyPrompt引入随机扩展的解析路由器实现实例级专家激活,并采用输出头的时间集成动态调整决策边界。理论与实证评估表明,该方法在CIFAR-100、ImageNet-R和CUB-200上分别相较最优基线提升11.23%、12.43%和7.62%。代码已开源。
原文摘要 · Abstract (English)
General continual learning (GCL) challenges intelligent systems to learn from single-pass, non-stationary data streams without clear task boundaries. While recent advances in continual parameter-efficient tuning (PET) of pretrained models show promise, they typically rely on multiple training epochs and explicit task cues, limiting their effectiveness in GCL scenarios. Moreover, existing methods often lack targeted design and fail to address two fundamental challenges in continual PET: how to allocate expert parameters to evolving data distributions, and how to improve their representational capacity under limited supervision. Inspired by the fruit fly's hierarchical memory system characterized by sparse expansion and modular ensembles, we propose FlyPrompt, a brain-inspired framework that decomposes GCL into two subproblems: expert routing and expert competence improvement. FlyPrompt introduces a randomly expanded analytic router for instance-level expert activation and a temporal ensemble of output heads to dynamically adapt decision boundaries over time. Extensive theoretical and empirical evaluations demonstrate FlyPrompt's superior performance, achieving up to 11.23%, 12.43%, and 7.62% gains over state-of-the-art baselines on CIFAR-100, ImageNet-R, and CUB-200, respectively. Our source code is available at https://github.com/AnAppleCore/FlyGCL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。