提出概率融合框架，提升混合正则化在条件密度估计中的效果
Mixup Regularization: A Probabilistic Perspective
- 基于对数线性池化实现概率分布的解析融合
- 在指数族数据上理论证明可精确计算联合似然
- 支持网络中间层输入融合，适用于复杂建模任务
近年来，混合法正则化通过训练数据的凸组合提升了深度学习模型的泛化能力。尽管已出现多种变体，其在条件密度估计与概率机器学习中的应用仍不充分。本文提出一种基于概率融合的新框架，更适用于条件密度估计任务。对于指数族分布的数据，我们证明可通过对数线性池化实现似然函数的解析融合。进一步提出扩展的随机混合方法，可在神经网络任意中间层融合输入。理论分析对比了该方法与标准混合法变体的差异。在合成与真实数据集上的实验结果表明，所提框架优于现有混合法变体。
原文摘要 · Abstract (English)
In recent years, mixup regularization has gained popularity as an effective way to improve the generalization performance of deep learning models by training on convex combinations of training data. While many mixup variants have been explored, the proper adoption of the technique to conditional density estimation and probabilistic machine learning remains relatively unexplored. This work introduces a novel framework for mixup regularization based on probabilistic fusion that is better suited for conditional density estimation tasks. For data distributed according to a member of the exponential family, we show that likelihood functions can be analytically fused using log-linear pooling. We further propose an extension of probabilistic mixup, which allows for fusion of inputs at an arbitrary intermediate layer of the neural network. We provide a theoretical analysis comparing our approach to standard mixup variants. Empirical results on synthetic and real datasets demonstrate the benefits of our proposed framework compared to existing mixup variants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。