arXiv:2504.17069cs.CVcs.AI2025-04被引 1

用自蒸馏选出最优生成顺序,提升图像生成质量同时保持灵活性。

Distilling Specialized Orders for Visual Generation

  • 先训练任意顺序模型,再从置信度中提取专用生成顺序。
  • ImageNet上FID从2.39降至2.17,零样本修复和扩展无需重训。
  • 轻量微调即可,适合追求高效与通用性的视觉生成研究者。

自回归(AR)图像生成器因生成质量高且可扩展而日益流行。传统AR模型固定采用从左上到右下的光栅扫描顺序,导致无法灵活支持修复、编辑、外扩等任务,需重新训练。任意顺序的AR模型虽能解决此问题,但复杂度高、性能下降。本文提出有序自回归(OAR)生成,通过自蒸馏流程:先训练任意顺序模型,再根据其置信度得分提取专用生成顺序,并在此顺序上微调。该方法实现两个目标:1)将模型容量从学习全部 $N!$ 种顺序转向单一专用路径,提升生成质量;2)保留任意顺序模型的多任务灵活性。在 ImageNet $256 imes 256$ 上,OAR将FID从2.39降至2.17,对Fashion Products和CelebA-HQ也保持一致提升。OAR支持零样本修复与外扩,人类评估显示64%偏好优于基线。该流程仅需对预训练任意顺序模型进行轻量微调,无需架构改动或额外标注。

原文摘要 · Abstract (English)

Autoregressive (AR) image generators are becoming increasingly popular due to their ability to produce high-quality images and their scalability. Typical AR models are locked onto a specific generation order, often a raster-scan from top-left to bottom-right; this prohibits multi-task flexibility (inpainting, editing, outpainting) without retraining. Any-order AR models address this by learning to generate under arbitrary patch orderings, but at the cost of increased complexity and lower performance. In this paper, we present Ordered Autoregressive (OAR) generation, a self-distillation pipeline that first trains an any-order AR model, then extracts specialized generation orders from the model's own confidence scores, and fine-tunes on these orders. This achieves two goals: 1) improved generation quality by redirecting capacity from learning all $N!$ orderings to a single specialized path, and 2) preserved flexibility of any-order models. On ImageNet $256\times 256$, OAR improves FID from 2.39 to 2.17 over the any-order baseline, with consistent gains on Fashion Products and CelebA-HQ. OAR supports zero-shot inpainting and outpainting without retraining, and human evaluation shows 64% preference over the baseline. The pipeline requires only lightweight fine-tuning on a pretrained any-order model, with no architectural changes or additional annotations.

图像生成自回归自蒸馏灵活生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。