动态扩展专家模型，让生成式检索在不重训的情况下高效适应新文档。
MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
- 用可扩展的低秩专家混合架构，按需添加新专家。
- 在两个数据集上性能优于全模型更新，参数增长近乎线性以下。
- 适合资源有限、需持续更新的生成式检索系统使用。
在资源受限条件下，持续用新文档更新基于模型的生成式检索索引仍具挑战性，因完整重训练计算成本过高。本文提出 MixLoRA-DSI，一种结合可扩展低秩适配（LoRA）专家混合与逐层分布外（OOD）驱动扩展策略的新框架。不同于为每个新语料库分配新专家，该策略仅在检测到大量分布外文档时，选择性引入新专家，实现亚线性参数增长。在 NQ320k 与 MS MARCO Passage 数据集上的实验表明，MixLoRA-DSI 性能超越全模型更新基线，参数开销极小，训练成本显著降低。
原文摘要 · Abstract (English)
Continually updating model-based indexes in generative retrieval with new documents remains challenging, as full retraining is computationally expensive and impractical under resource constraints. We propose MixLoRA-DSI, a novel framework that combines an expandable mixture of Low-Rank Adaptation experts with a layer-wise out-of-distribution (OOD)-driven expansion strategy. Instead of allocating new experts for each new corpus, our proposed expansion strategy enables sublinear parameter growth by selectively introducing new experts only when significant number of OOD documents are detected. Experiments on NQ320k and MS MARCO Passage demonstrate that MixLoRA-DSI outperforms full-model update baselines, with minimal parameter overhead and substantially lower training costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。