无需量化直接生成图像，用评分最大化实现连续视觉自回归建模
Continuous Visual Autoregressive Generation via Score Maximization
- 基于严格合适评分规则，直接在连续空间训练图像生成
- 采用能量评分作为目标函数，无需概率密度估计即可优化
- 统一框架可推导出扩散模型等方法，适合生成模型研究者
传统观念认为自回归模型仅适用于离散数据。处理图像等连续模态时，视觉自回归（VAR）通常依赖量化将数据转为离散表示，导致显著信息损失。为此，本文提出连续VAR框架，实现无需向量量化即可直接生成视觉数据。其理论基础为严格合适评分规则，提供强大统计工具以评估生成模型对真实分布的逼近程度。只需选择一个严格合适评分并设为训练目标即可优化模型。本文重点探索基于能量评分的训练目标，该方法无需似然计算，克服了连续空间中概率预测的困难。此前如GIVT和扩散损失等连续自回归生成方法，也可通过选用其他严格合适评分从本框架推导得出。代码已开源：https://github.com/shaochenze/EAR。
原文摘要 · Abstract (English)
Conventional wisdom suggests that autoregressive models are used to process discrete data. When applied to continuous modalities such as visual data, Visual AutoRegressive modeling (VAR) typically resorts to quantization-based approaches to cast the data into a discrete space, which can introduce significant information loss. To tackle this issue, we introduce a Continuous VAR framework that enables direct visual autoregressive generation without vector quantization. The underlying theoretical foundation is strictly proper scoring rules, which provide powerful statistical tools capable of evaluating how well a generative model approximates the true distribution. Within this framework, all we need is to select a strictly proper score and set it as the training objective to optimize. We primarily explore a class of training objectives based on the energy score, which is likelihood-free and thus overcomes the difficulty of making probabilistic predictions in the continuous space. Previous efforts on continuous autoregressive generation, such as GIVT and diffusion loss, can also be derived from our framework using other strictly proper scores. Source code: https://github.com/shaochenze/EAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。