arXiv:2603.15708cs.LGcs.AI2026-03

用不确定性引导专家协作,提升长尾序列识别能力。

Mastering the Minority: An Uncertainty-guided Multi-Expert Framework for Challenging-tailed Sequence Learning

  • 引入基于证据理论的专家分工机制,实现对少数类的有效建模。
  • 在四个数据集上最高提升17.97%的分类性能,参数量减少10.32%。
  • 适合处理长尾分布的文本分类任务,尤其对少数类别敏感场景有效。

序列学习中的数据分布不均问题仍是一个关键挑战,模型容易识别高频类别却难以检测少数类别。混合专家模型虽具可扩展性,但常受参数效率低、专家分化不足和预测冲突难解等问题制约。为有效应对少数类识别难题,我们提出不确定性引导的多专家融合网络(UME)。UME包含三项核心创新:首先,采用集成LoRA实现参数高效建模,显著降低可训练参数量;其次,引入基于德普斯特-谢弗理论(DST)的序列化专家分工机制,确保对长尾类别的有效专化;最后,设计不确定性引导融合机制,利用DST的确定性度量动态加权专家意见,通过优先采纳最自信专家来解决冲突,保障最终预测可靠性。在四个公开层级文本分类数据集上的大量实验表明,UME达到当前最优性能,在个别类别上相较最佳基线提升高达17.97%,同时可训练参数减少最多达10.32%。研究结果表明,不确定性引导的专家协同是应对长尾序列学习的合理策略。代码已开源:https://github.com/CQUPTWZX/Multi-experts。

原文摘要 · Abstract (English)

Imbalanced data distribution remains a critical challenge in sequential learning, leading models to easily recognize frequent categories while failing to detect minority classes adequately. The Mixture-of-Experts model offers a scalable solution, yet its application is often hindered by parameter inefficiency, poor expert specialization, and difficulty in resolving prediction conflicts. To Master the Minority classes effectively, we propose the Uncertainty-based Multi-Expert fusion network (UME) framework. UME is designed with three core innovations: First, we employ Ensemble LoRA for parameter-efficient modeling, significantly reducing the trainable parameter count. Second, we introduce Sequential Specialization guided by Dempster-Shafer Theory (DST), which ensures effective specialization on the challenging-tailed classes. Finally, an Uncertainty-Guided Fusion mechanism uses DST's certainty measures to dynamically weigh expert opinions, resolving conflicts by prioritizing the most confident expert for reliable final predictions. Extensive experiments across four public hierarchical text classification datasets demonstrate that UME achieves state-of-the-art performance. We achieve a performance gain of up to 17.97\% over the best baseline on individual categories, while reducing trainable parameters by up to 10.32\%. The findings highlight that uncertainty-guided expert coordination is a principled strategy for addressing challenging-tailed sequence learning. Our code is available at https://github.com/CQUPTWZX/Multi-experts.

长尾学习多专家不确定性文本分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。