提出RankE框架,让离散文本到图像生成的编码器与解码器共同进化,提升图像质量。
RankE: End-to-End Post-Training for Discrete Text-to-Image Generation with Decoder Co-Evolution

- 通过交替优化策略与解码器,实现编码器-解码器协同进化
- 在MS-COCO 30K上,FID降至15.21,CLIP得分提升至33.76
- 适合追求高质量离散生成的科研与工程人员
离散自回归(AR)文本到图像(T2I)模型使用VQ分词器与AR策略,现有后训练流程仅优化策略而冻结解码器。近期扩散模型研究显示,VAE是关键对齐瓶颈,但离散AR模型尚未有类似探索。我们发现仅优化策略会导致潜在变量协变量偏移:随着策略演化,生成标记分布偏离解码器训练时的真值分布,导致奖励分数上升但图像质量下降。为此,我们提出首个端到端后训练框架RankE,不固定解码器,而是通过交替优化使两个组件共同进化:每个模块在基于排序的对齐目标下最大化性能,并受适配其参数空间的稳定性锚点正则化。该协同进化打破冻结解码器方法中的保真度-对齐权衡——在LlamaGen-XL(775M)上,标准强化学习虽提升CLIP得分却恶化FID,而RankE同时提升两者(在MS-COCO 30K上FID 15.21,CLIP 33.76)。Janus-Pro(1B)上的稳定增益证实,解码器协同进化可将奖励优化可靠转化为像素级质量提升。
原文摘要 · Abstract (English)
Discrete autoregressive (AR) text-to-image (T2I) models pair a VQ tokenizer with an AR policy, and current post-training pipelines optimize only the policy while keeping the VQ decoder frozen. Recent diffusion T2I work, exemplified by REPA-E, has shown that the VAE itself constitutes a key alignment bottleneck, yet no analogous investigation exists for discrete AR models. We show that policy-only optimization induces Latent Covariate Shift: as the policy evolves, the resulting token distribution diverges from the ground-truth distribution on which the decoder was trained, such that reward scores improve while decoded image quality degrades. To address this mismatch, we propose RankE, the first end-to-end post-training framework for discrete T2I generation. Rather than optimizing the policy against a fixed decoder, RankE co-evolves both components through alternating optimization: each module maximizes a ranking-based alignment objective while being regularized by a stability-preserving anchor suited to its parameter space. This co-evolution breaks the fidelity--alignment trade-off that plagues frozen-decoder approaches: on LlamaGen-XL (775M), standard RL improves CLIP but degrades FID, whereas RankE improves both simultaneously (FID 15.21, CLIP 33.76 on MS-COCO 30K). Consistent gains on Janus-Pro (1B) confirm that decoder co-evolution reliably converts reward optimization into pixel-space quality improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。