提出新基准FLEX，揭示多模态持续学习中专家路由饱和问题
Beyond Routing Saturation: A Long-Horizon Class-Incremental Perspective on Expert Routing in Multimodal Continual Instruction Tuning
- 构建34任务长时序基准FLEX，弱化文本指纹干扰
- 首次发现专家路由在主流数据集上已接近饱和
- 将路由问题形式化为类增量学习，适配多种方法
多模态持续指令微调（MCIT）使多模态大模型能顺序学习新任务并保留旧知识。现有方法通常维护任务专属的LoRA专家，并在推理时路由输入至一个或多个专家。然而，专家路由背后的任务识别问题仍未被充分探索。我们发现，在广泛使用的MCIT基准上，路由几乎已达饱和状态。文本指纹泄露任务身份，以及仅4–10个任务、竞争专家稀少的短序列，共同掩盖了长时序路由挑战。为此，我们提出FLEX（Fingerprint-reduced Long-horizon Expert eXamination），一个包含34个任务的长时序MCIT基准，弱化了文本指纹。FLEX将具有相似指令和答案格式但视觉与知识领域不同的任务分组，标准化外层模板，并在更大规模的专家池上评估路由性能。关键的是，我们将渐进式LoRA路由形式化为软任务即类的多模态类增量学习（MCIL）：每个任务定义一个增量路由类别，其完整得分分布提供LoRA混合权重，硬路由则为离散特例。FLEX揭示了这一不断扩展的任务识别挑战，而MCIL框架为迁移类增量学习方法到专家路由提供了理论接口。我们以PureLoRA为对照基线，将四种类增量学习方法适配至四种MCIT框架，无需修改其LoRA专家或生成流程。所提出的插件式路由模块使严格LoRA匹配提升最高达16.3个百分点，整体MacroScore提升最高达4.6点。代码已开源。
原文摘要 · Abstract (English)
Multimodal Continual Instruction Tuning (MCIT) enables multimodal large language models to acquire new tasks sequentially while retaining previously learned capabilities. Many recent methods maintain task-specific LoRA experts and route each input to one or more experts at inference. Yet the task-identification problem underlying expert routing remains under-explored. We show that routing is nearly saturated on widely used MCIT benchmarks. Textual fingerprints that leak task identity and short 4--10-task sequences with few competing experts jointly obscure the long-horizon routing problem. To expose this challenge, we introduce FLEX (Fingerprint-reduced Long-horizon Expert eXamination), a 34-task long-horizon MCIT benchmark with weakened textual fingerprints. FLEX groups tasks with similar instruction and answer formats but diverse visual and knowledge domains, normalizes their outer templates, and evaluates routing over a substantially larger expert pool. Crucially, we formulate progressive-LoRA routing as soft task-as-class Multimodal Class-Incremental Learning (MCIL): each task defines an incremental routing class, whose complete score distribution supplies the LoRA mixture weights, with hard routing as a discrete special case. FLEX exposes this expanding task-identification challenge, while the MCIL formulation provides a principled interface for transferring CIL methods to expert routing. We instantiate PureLoRA as a controlled baseline and adapt four CIL methods to four MCIT frameworks without modifying their LoRA experts or generation pipelines. Our plug-in routers improve strict LoRA matching by up to 16.3 percentage points and overall MacroScore by up to 4.6 points. Code is available at: https://github.com/RINC-CL/FLEX
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。