提出信号空间对齐方法,提升二值数据生成的稳定性与鲁棒性。
Binary Flow Matching: Prediction-Loss Space Alignment for Robust Learning
- 通过信号空间对齐,解决二值生成中梯度敏感问题。
- 使用均匀采样时仍可稳定训练,无需依赖启发式调度。
- 适用于二值及离散数据生成,为扩散模型提供理论支撑。
流匹配已成为生成建模的强大框架,近期实证成功凸显了信号空间预测(x-预测)的有效性。本文研究该范式在二值流形上的迁移,这是离散数据生成的基础设置。尽管x-预测仍有效,但发现其与基于速度的目标(v-损失)耦合时存在潜在结构不匹配,导致时间依赖的奇异加权,放大近似误差带来的梯度敏感性。为此,我们形式化提出预测-损失对齐作为流匹配训练的必要条件。证明将目标重对齐至信号空间(x-损失)可消除奇异加权,获得一致有界的梯度,从而在均匀时间步采样下实现稳健训练,无需依赖启发式调度。最后,在对齐基础上,分析二值数据特有设计选择,揭示概率目标(如交叉熵)与几何损失(如均方误差)之间的拓扑依赖差异。这些结果为二值及类似离散域的稳健流匹配提供了理论基础与实践指导,确立信号空间对齐为稳健扩散学习的关键原则。
原文摘要 · Abstract (English)
Flow matching has emerged as a powerful framework for generative modeling, with recent empirical successes highlighting the effectiveness of signal-space prediction ($x$-prediction). In this work, we investigate the transfer of this paradigm to binary manifolds, a fundamental setting for generative modeling of discrete data. While $x$-prediction remains effective, we identify a latent structural mismatch that arises when it is coupled with velocity-based objectives ($v$-loss), leading to a time-dependent singular weighting that amplifies gradient sensitivity to approximation errors. Motivated by this observation, we formalize prediction-loss alignment as a necessary condition for flow matching training. We prove that re-aligning the objective to the signal space ($x$-loss) eliminates the singular weighting, yielding uniformly bounded gradients and enabling robust training under uniform timestep sampling without reliance on heuristic schedules. Finally, with alignment secured, we examine design choices specific to binary data, revealing a topology-dependent distinction between probabilistic objectives (e.g., cross-entropy) and geometric losses (e.g., mean squared error). Together, these results provide theoretical foundations and practical guidelines for robust flow matching on binary -- and related discrete -- domains, positioning signal-space alignment as a key principle for robust diffusion learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。