通过分解潜在空间提升LoRA的适配性能，让模型更专注关键信息。
Latent Space Factorization in LoRA
- 用变分自编码器构建两个分离的潜在空间，分别捕捉任务相关与残留特征。
- 在文本、音频、图像任务上均优于标准LoRA，分布外场景下鲁棒性更强。
- 适合需要高精度微调且关注特征解耦的科研与工程人员。
低秩适应（LoRA）是一种广泛使用的参数高效微调方法。然而，现有LoRA变体缺乏显式机制来区分低秩子空间中的任务相关信号，可能限制下游性能。本文提出因子化变分自编码器LoRA（FVAE-LoRA），利用变分自编码器学习两个独立的潜在空间。我们提出的证据下界（ELBO）公式显式促进两空间间的解耦，一个专用于任务关键特征，另一个处理残差信息。在文本、音频和图像任务上的大量实验表明，FVAE-LoRA持续优于标准LoRA。此外，虚假相关性评估显示，FVAE-LoRA能更好隔离任务相关信号，在分布偏移下表现更稳健。代码已公开：https://github.com/idiap/FVAE-LoRA
原文摘要 · Abstract (English)
Low-rank adaptation (LoRA) is a widely used method for parameter-efficient finetuning. However, existing LoRA variants lack mechanisms to explicitly disambiguate task-relevant information within the learned low-rank subspace, potentially limiting downstream performance. We propose Factorized Variational Autoencoder LoRA (FVAE-LoRA), which leverages a VAE to learn two distinct latent spaces. Our novel Evidence Lower Bound formulation explicitly promotes factorization between the latent spaces, dedicating one latent space to task-salient features and the other to residual information. Extensive experiments on text, audio, and image tasks demonstrate that FVAE-LoRA consistently outperforms standard LoRA. Moreover, spurious correlation evaluations confirm that FVAE-LoRA better isolates task-relevant signals, leading to improved robustness under distribution shifts. Our code is publicly available at: https://github.com/idiap/FVAE-LoRA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。