用分布距离优化自回归图像生成，显著提升图像质量。
Beyond Token-Level Cross-Entropy: Fréchet Distributional Post-Training for Autoregressive Image Generation
- 后训练阶段用表示空间弗雷歇距离作为唯一目标
- 在不增加参数和推理步数下，FID降低41.4%
- 适合希望提升图像生成质量的开发者
自回归图像生成器通常在教师强制下使用词元级交叉熵预训练，但评估时依赖解码图像的分布质量。这导致目标不匹配：分类错误对图像的影响不同；且推理依赖模型生成的历史，存在上下文差异。本文提出FD-loss后训练方法，仅以表示空间弗雷歇距离为优化目标，适配预训练离散生成器。采用双遍方案：第一遍在模型原生推理配置下无梯度生成，构建脱离的回放上下文；第二遍通过概率级直通估计器（STE）进行可微重播，前向保持硬argmax解码，反向传播温度缩放后的概率梯度。仅更新生成器，分词器与特征提取器冻结。在四类生成器家族、八种配置下，于256×256类别条件ImageNet上，平均使FID和FD_r6分别降低41.4%和52.0%。最强结果从2.42降至1.43，未增加参数或推理步数。
原文摘要 · Abstract (English)
Autoregressive image generators are commonly pretrained with token-level cross-entropy under teacher forcing, yet evaluated by the distributional quality of decoded images. This creates an objective mismatch, because categorical errors have unequal image-level consequences, and a context mismatch, because inference conditions on model-generated histories. We introduce FD-loss post-training, which adapts a pretrained discrete generator using representation-space Fréchet distance as the sole objective. A dual-pass scheme first constructs detached rollout contexts through gradient-free generation under the model's native inference configuration, then performs differentiable replay with a probability-level straight-through estimator (STE) that preserves hard argmax decoding in the forward pass while propagating image-level gradients through temperature-scaled probabilities. Only the generator is updated, while the tokenizer and feature extractors remain frozen. Across eight completed configurations from four generator families on class-conditional ImageNet at $256\times256$, FD-loss post-training reduces FID and $\mathrm{FD}_{r6}$ by 41.4% and 52.0% on average. The strongest FID result improves from 2.42 to 1.43 without adding parameters or inference steps.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。