用贝叶斯方法自动优化数据增强参数,提升模型鲁棒性。
Optimizing Data Augmentation through Bayesian Model Selection
- 将数据增强参数视为超参数,通过边际似然优化
- 在视觉和NLP任务上显著提升校准性能与泛化能力
- 适合追求模型稳定性和自动化调参的研究者
数据增强已成为提升现代机器学习模型鲁棒性与泛化能力的关键工具。然而,选择合适的增强策略及其参数仍需谨慎,传统方法依赖试错或基于验证性能的昂贵优化。本文提出一种新框架,从概率视角看待数据增强,将增强参数视为模型(超)参数,并将其优化转化为贝叶斯模型选择问题。由于该问题不可解析求解,我们推导出一个可计算的变分下界(ELBO),实现增强参数与模型参数的联合优化。理论分析涵盖变分近似质量、泛化保证、不变性性质及与经验贝叶斯的联系。在计算机视觉与自然语言处理任务上的实验表明,该方法显著改善模型校准性,并在固定或无增强情况下表现更稳健。本工作为基于贝叶斯原则优化数据增强提供了严谨基础,具有推动鲁棒机器学习的潜力。
原文摘要 · Abstract (English)
Data Augmentation (DA) has become an essential tool to improve robustness and generalization of modern machine learning. However, when deciding on DA strategies it is critical to choose parameters carefully, and this can be a daunting task which is traditionally left to trial-and-error or expensive optimization based on validation performance. In this paper, we counter these limitations by proposing a novel framework for optimizing DA. In particular, we take a probabilistic view of DA, which leads to the interpretation of augmentation parameters as model (hyper)-parameters, and the optimization of the marginal likelihood with respect to these parameters as a Bayesian model selection problem. Due to its intractability, we derive a tractable ELBO, which allows us to optimize augmentation parameters jointly with model parameters. We provide extensive theoretical results on variational approximation quality, generalization guarantees, invariance properties, and connections to empirical Bayes. Through experiments on computer vision and NLP tasks, we show that our approach improves calibration and yields robust performance over fixed or no augmentation. Our work provides a rigorous foundation for optimizing DA through Bayesian principles with significant potential for robust machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。