arXiv:2602.06862cs.CV2026-02中稿 · ICML被引 1

用动态参数路由让模型专家分工,高效适配视觉任务。

Parameters as Experts: Adapting Vision Models with Dynamic Parameter Routing

  • 引入专家中心与动态路由,按输入选择性组合参数矩阵。
  • 在多个视觉任务上逼近全量微调效果,仅需极少可训练参数。
  • 适合需要轻量化微调的场景,如资源受限部署或多任务适配。

使用参数高效微调(PEFT)适配预训练视觉模型仍具挑战,目标是在仅使用少量可训练参数的情况下达到与全量微调相当的性能。现有方法在复杂密集预测任务中存在输入无关建模和跨层表示冗余等问题。为此,我们提出ParaX,一种基于混合专家(MoE)架构的适配器方法。具体而言,引入共享专家中心,每个专家为一个可训练参数矩阵。前向传播时,网络中每个ParaX模块通过简单动态参数路由机制,为当前模块生成定制化权重矩阵,动态聚合对应专家中心中的参数矩阵。这种动态权重矩阵实现输入依赖的低秩适配,生成更个性化且强大的特征表示。此外,由于多层ParaX模块共享同一专家中心,促进隐式跨层特征交互,提升特征多样性。大量实验表明,ParaX在多种视觉识别任务中表现优异。代码已公开:https://github.com/LMMMEng/ParaX。

原文摘要 · Abstract (English)

Adapting pre-trained vision models using parameter-efficient fine-tuning (PEFT) remains challenging, as it aims to achieve performance comparable to full fine-tuning using a minimal number of trainable parameters. When applied to complex dense prediction tasks, existing methods exhibit limitations, including input-agnostic modeling and redundant cross-layer representations. To this end, we propose ParaX, a new adapter-style method featuring a simple mixture-of-experts (MoE) architecture. Specifically, we introduce shared expert centers, where each expert is a trainable parameter matrix. During a feedforward pass, each ParaX module in the network dynamically generates weight matrices tailored for the current module via a simple dynamic parameter routing mechanism, which selectively aggregates parameter matrices in the corresponding expert center. Dynamic weight matrices in ParaX modules facilitate low-rank adaptation in an input-dependent manner, thus generating more customized and powerful feature representations. Moreover, since ParaX modules across multiple network layers share the same expert center, they improve feature diversity by promoting implicit cross-layer feature interaction. Extensive experimental results demonstrate the superiority of ParaX across diverse visual recognition tasks. Code is publicly released at: https://github.com/LMMMEng/ParaX.

参数高效视觉模型动态路由MoE

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。