arXiv:2508.01598cs.LGcs.AI2025-08AAAI被引 26

动态协作专家混合模型,应对多源数据流的异构与概念漂移问题。

Drift-aware Collaborative Assistance Mixture of Experts for Heterogeneous Multistream Learning

  • 为每条数据流配备专属特征提取器和专家,实现异构性建模。
  • 通过辅助专家自适应融合其他流的相关信息,有效防止负迁移。
  • 支持专家生命周期自动管理,动态适应概念漂移,适合在线学习场景。

现实场景中多数据流学习面临固有的异构性和不可预测的概念漂移,现有方法通常假设流同质且采用静态架构,知识融合无差别,限制了在复杂动态环境中的泛化能力。为此,我们提出CAMEL框架,一种动态协同辅助专家混合学习机制。该框架为每条流配置独立系统,包含专用特征提取器和任务特定头,并引入一个动态私有专家池以捕捉各流特有的模式。关键在于,通过专门的辅助专家,利用多头注意力机制从所有并发流中自主提炼并集成相关上下文,实现精准知识迁移,同时天然抑制无关源带来的负迁移。此外,我们提出自主专家调优策略(AET),根据概念漂移动态管理专家生命周期:对新出现概念实例化新专家(冻结旧专家以避免灾难性遗忘),并删除过时专家。这种专家级可塑性提供了鲁棒高效的在线模型容量自适应机制。大量实验表明,CAMEL在多种多流场景下具有卓越的泛化能力,并对复杂概念漂移表现出极强的鲁棒性。

原文摘要 · Abstract (English)

Learning from multiple data streams in real-world scenarios is fundamentally challenging due to intrinsic heterogeneity and unpredictable concept drifts. Existing methods typically assume homogeneous streams and employ static architectures with indiscriminate knowledge fusion, limiting generalizability in complex dynamic environments. To tackle this gap, we propose CAMEL, a dynamic \textbf{C}ollaborative \textbf{A}ssistance \textbf{M}ixture of \textbf{E}xperts \textbf{L}earning framework. It addresses heterogeneity by assigning each stream an independent system with a dedicated feature extractor and task-specific head. Meanwhile, a dynamic pool of specialized private experts captures stream-specific idiosyncratic patterns. Crucially, collaboration across these heterogeneous streams is enabled by a dedicated assistance expert. This expert employs a multi-head attention mechanism to distill and integrate relevant context autonomously from all other concurrent streams. It facilitates targeted knowledge transfer while inherently mitigating negative transfer from irrelevant sources. Furthermore, we propose an Autonomous Expert Tuner (AET) strategy, which dynamically manages expert lifecycles in response to drift. It instantiates new experts for emerging concepts (freezing prior ones to prevent catastrophic forgetting) and prunes obsolete ones. This expert-level plasticity provides a robust and efficient mechanism for online model capacity adaptation. Extensive experiments demonstrate CAMEL's superior generalizability across diverse multistreams and exceptional resilience against complex concept drifts.

多流学习概念漂移专家混合在线学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。