通过软对数变换让生成模型更好处理长尾数据
Tail Annealing for Heavy-Tailed Flow Matching
- 对数据逐维应用软对数变换，再指数还原生成结果
- 在144个配置上显著优于专用基线，极端分位数指标最优
- 无需修改架构或分布，适合长尾数据生成任务
标准生成模型难以处理长尾数据：Lipschitz架构无法从高斯噪声生成幂律尾部，且在长尾数据与高斯分布间插值是病态问题。我们提出简单解决方案：训练前对数据逐维应用软对数变换ϕ(x) = sign(x)·log(1+|x|)，生成时再指数还原。通过Hill诊断决定每维是否变换，轻尾部分保持不变，无额外复杂度。该方法将长尾压缩至标准流匹配可处理范围，无需长尾基分布或架构修改。理论分析表明，对数变换将帕累托尾部映射为指数分布，诱导动态实现幂次变换形式的尾部退火。在包含3种协变结构、维度d最高100、4种尾指数的144配置多变量基准测试中，Log-FM在W₁、CVaR₉₉及极端分位数指标上全面领先专用基线，且在2,880次运行中零严重发散。
原文摘要 · Abstract (English)
Standard generative models struggle with heavy-tailed data: Lipschitz architectures cannot produce power-law tails from Gaussian noise, and interpolating between heavy-tailed data and Gaussians is ill-posed. We propose a simple fix: apply the soft-log transform $ϕ(x) = \mathrm{sign}(x) \cdot \log(1 + |x|)$ coordinate-wise to data before training, then exponentiate samples after generation. A Hill diagnostic decides per-coordinate whether to transform, leaving light-tailed margins untouched at no added complexity. This compresses heavy tails into a range where standard flow matching succeeds, without heavy-tailed base distributions or architectural modifications. We provide theoretical intuition for why this works: the log-transform maps Pareto tails to exponentials, and the induced dynamics implement a form of tail annealing via power transformations. On a 144-configuration multivariate benchmark (3 copulas, $d$ up to 100, 4 tail indices), Log-FM dominates specialized baselines on $W_1$, CVaR$_{99}$, and extreme-quantile metrics, and is the only method with zero severe divergences across 2{,}880 runs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。