改进流匹配的损失函数,让模型更关注罕见或复杂分支的数据结构。
Risk-Entropic Flow Matching
- 用对数指数变换改造损失函数,增强对稀有事件的关注。
- 在合成数据上显著提升几何结构恢复能力,统计指标更优。
- 适合处理具有多模态、长尾分布的数据,如医学影像或异常检测。
倾斜(熵化)风险通过在基础损失上应用对数指数变换,是统计与机器学习中用于强调罕见或高损失事件的成熟工具,同时保持优化问题的可解性。本文旨在解析其在流匹配(Flow Matching, FM)中的结构。标准的修正流匹配通过线性插值源样本与数据样本生成训练对,并使用均方误差损失训练神经速度场以预测直线位移。然而,这种平方损失将所有到达同一时空点的速度目标合并为单一条件均值,忽略了方差、偏度、多模态等高阶条件信息,从而丢失了数据流形的精细几何结构和少数分支特征。本文将标准的风险敏感(对数指数)变换应用于条件流匹配损失,证明所得倾斜风险损失是每个时空点上一个有意义的条件熵流匹配目标的自然上界。进一步,我们发现该条件熵目标梯度的小阶展开包含两个可解释的一阶修正项:协方差预处理的流残差项,以及偏好非对称或稀有分支的偏度尾部项。在设计用于探测模糊性和尾部行为的合成数据上,所提出的风险敏感损失在统计指标上优于标准修正流匹配,并更准确地恢复几何结构。
原文摘要 · Abstract (English)
Tilted (entropic) risk, obtained by applying a log-exponential transform to a base loss, is a well established tool in statistics and machine learning for emphasizing rare or high loss events while retaining a tractable optimization problem. In this work, our aim is to interpret its structure for Flow Matching (FM). FM learns a velocity field that transports samples from a simple source distribution to data by integrating an ODE. In rectified FM, training pairs are obtained by linearly interpolating between a source sample and a data sample, and a neural velocity field is trained to predict the straight line displacement using a mean squared error loss. This squared loss collapses all velocity targets that reach the same space-time point into a single conditional mean, thereby ignoring higher order conditional information (variance, skewness, multi-modality) that encodes fine geometric structure about the data manifold and minority branches. We apply the standard risk-sensitive (log-exponential) transform to the conditional FM loss and show that the resulting tilted risk loss is a natural upper-bound on a meaningful conditional entropic FM objective defined at each space-time point. Furthermore, we show that a small order expansion of the gradient of this conditional entropic objective yields two interpretable first order corrections: covariance preconditioning of the FM residual, and a skew tail term that favors asymmetric or rare branches. On synthetic data designed to probe ambiguity and tails, the resulting risk-sensitive loss improves statistical metrics and recovers geometric structure more faithfully than standard rectified FM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。