通过交互式贝叶斯机制提升大模型微调时的集成多样性。
Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-tuning Foundation Models
- 引入交互式贝叶斯分布鲁棒性框架，建模粒子间交互以增强多样性。
- 在VTAB-1K和通用推理任务上优于多个基线方法。
- 适合需要高鲁棒性和多样性的大模型微调场景。
我们提出交互式贝叶斯分布鲁棒性（IBDR），一种新型贝叶斯推断框架，能够建模粒子间的交互，从而通过提升粒子多样性来增强集成质量。IBDR基于一个广义理论框架，将分布群体损失与近似后验相联系，推动了一种实用的双优化流程，在保证分布鲁棒性的同时促进粒子多样性。我们在VTAB-1K基准和通用推理语言任务上评估了IBDR的性能，结果一致显示其优于多个基线方法，验证了其在真实应用中的有效性。
原文摘要 · Abstract (English)
We introduce Interactive Bayesian Distributional Robustness (IBDR), a novel Bayesian inference framework that allows modeling the interactions between particles, thereby enhancing ensemble quality through increased particle diversity. IBDR is grounded in a generalized theoretical framework that connects the distributional population loss with the approximate posterior, motivating a practical dual optimization procedure that enforces distributional robustness while fostering particle diversity. We evaluate IBDR's performance against various baseline methods using the VTAB-1K benchmark and the common reasoning language task. The results consistently show that IBDR outperforms these baselines, underscoring its effectiveness in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。