让多个视觉语言模型专家协同工作，不遗忘旧知识且推理快。
GeoStack: A Framework for Quasi-Abelian Knowledge Composition in VLMs
- 通过几何约束组合独立训练的领域专家模型
- 支持无限叠加专家，推理时间恒定（O(1））
- 适合长期累积多领域知识的场景
我们解决视觉语言模型（VLMs）中知识组合的挑战：跨领域或任务积累经验常导致灾难性遗忘。提出GeoStack（几何堆叠）框架，使独立训练的领域专家可组合为统一模型。通过对适配器流形施加几何与结构约束，确保基础模型知识不被破坏。进一步从数学上证明了权重折叠特性，实现无论集成多少专家，推理复杂度恒为O(1)。在多领域迁移与类增量学习实验中，GeoStack展现出高效长期知识组合能力，并显著缓解灾难性遗忘。代码已公开于https://github.com/QuantitativeImagingLaboratory/GeoStack。
原文摘要 · Abstract (English)
We address the challenge of knowledge composition in Vision-Language Models (VLMs), where accumulating expertise across multiple domains or tasks typically leads to catastrophic forgetting. We introduce GeoStack (Geometric Stacking), a modular framework that allows independently trained domain experts to be composed into a unified model. By imposing geometric and structural constraints on the adapter manifold, GeoStack ensures the foundational knowledge of the base model is preserved. Furthermore, we mathematically demonstrate a weight-folding property that achieves constant-time inference complexity ($O(1)$), regardless of the number of integrated experts. Experimental results across multi-domain adaptation and class-incremental learning show that GeoStack provides an efficient mechanism for long-term knowledge composition while significantly mitigating catastrophic forgetting. Code is available at https://github.com/QuantitativeImagingLaboratory/GeoStack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。