用图神经网络动态分配视觉块，实现轻量高效领域泛化。
GNN-MoE: Context-Aware Patch Routing using GNNs for Parameter-Efficient Domain Generalization
- 基于图神经网络构建跨图像块的路由机制，动态选择专家模块。
- 在多个领域泛化基准上达到顶尖性能，仅使用少量可训练参数。
- 适合追求轻量化部署的视觉模型跨域适应场景。
领域泛化（DG）旨在提升视觉变换器（ViT）在未见领域上的鲁棒性。高效适配预训练ViT进行DG极具挑战，标准微调成本高且可能损害泛化能力。本文提出GNN-MoE，通过混合专家（MoE）框架结合高效的克罗内克适配器，增强参数高效微调（PEFT）的DG能力。不同于传统的基于标记的路由，创新性地采用图神经网络（GCN、GAT、SAGE）对图像块间的互关系图进行建模，动态分配各块至特定专家。该上下文感知的图路由机制利用块间关联信息，更好地应对领域偏移。GNN-MoE在多个主流领域泛化基准上取得当前最优或竞争力表现，同时保持极高的参数效率，凸显了基于图的上下文路由在实现鲁棒、轻量级领域泛化中的有效性。
原文摘要 · Abstract (English)
Domain generalization (DG) seeks robust Vision Transformer (ViT) performance on unseen domains. Efficiently adapting pretrained ViTs for DG is challenging; standard fine-tuning is costly and can impair generalization. We propose GNN-MoE, enhancing Parameter-Efficient Fine-Tuning (PEFT) for DG with a Mixture-of-Experts (MoE) framework using efficient Kronecker adapters. Instead of token-based routing, a novel Graph Neural Network (GNN) router (GCN, GAT, SAGE) operates on inter-patch graphs to dynamically assign patches to specialized experts. This context-aware GNN routing leverages inter-patch relationships for better adaptation to domain shifts. GNN-MoE achieves state-of-the-art or competitive DG benchmark performance with high parameter efficiency, highlighting the utility of graph-based contextual routing for robust, lightweight DG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。