受果蝇记忆机制启发,提出新方法实现持续学习中旧知识不遗忘。
Toward Robust Open-set Adaptation: Synapse Consolidation Inspired by Rac1/MAPK Pathways
- 借鉴Rac1/MAPK通路设计双组件机制,分别控制更新范围与抑制噪声
- 在18个NLP数据集上对未见任务适应达78.31%,对分布漂移鲁棒性达85.37%
- 适合需要长期部署、任务持续变化的开放环境应用
大型语言模型虽具跨任务泛化能力,但在实际部署中面对持续演进的任务和分布漂移仍易失效。现有测试时自适应方法因忽视源知识保留与更新信号可靠性而受限。受果蝇通过Rac1/MAPK通路平衡前向与后向干扰以调控记忆更新的启发,本文提出同步固化(SyCo)方法,包含两个核心组件:受Rac1启发的可塑性约束器,动态将更新限制在对源知识影响较小的尾梯度子空间,实现快速专业化同时保留原始表示;受MAPK启发的更新控制器,采用分层策略抑制噪声更新,在非平稳数据流中巩固有效适应。为更贴近真实部署场景,引入多源开放集自适应(MOA)设定——模型在多个标注源任务上训练后,需在混合已知与未知任务、标签与意图部分重叠的无标签测试流中持续适应。在18个NLP数据集上的实验表明,SyCo显著优于强基线,未见任务适应准确率达78.31%,应对未见数据分布漂移时达85.37%,刷新当前最佳表现。
原文摘要 · Abstract (English)
Large Language Models (LLMs) generalize across tasks through reusable representations and flexible reasoning, yet remain brittle in real deployment when faced with evolving tasks and continual distribution shift. While test-time adaptation addresses this by updating models with unsupervised objectives on test data, prevailing methods are fundamentally limited by their neglect of source knowledge preservation and adaptation signal reliability. Inspired by how Drosophila orchestrates memory update by balancing retroactive and proactive interference via Rac1 and MAPK pathways, we design Synapse Consolidation (SyCo) with two core components: a Rac1-inspired plasticity confiner and a MAPK-inspired update controller. The former dynamically confines plasticity to a tail-gradient subspace that is less critical for source knowledge, enabling rapid specialization while preserving source representations. The latter uses a tiered controller to suppress noisy updates and consolidate useful adaptations under non-stationary streams. To further model real deployments with multiple sources and continually emerging tasks, we introduce Multi-source Open-set Adaptation (MOA) setting, where a model is trained on multiple labeled source tasks and then adapts on open, non-stationary unlabeled test streams mixing seen and unseen tasks with partial overlap in label and intent space. Across 18 NLP datasets under the MOA setting, SyCo consistently outperforms strong baselines, achieving 78.31\% on unseen-task adaptation and 85.37\% versus unseen-data shifts, setting a new state-of-the-art.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。