arXiv:2605.03317cs.CVcs.AI2026-05被引 1

动态调整扩散模型的对齐粒度,提升生成质量和训练效率

AHPA: Adaptive Hierarchical Prior Alignment for Diffusion Transformers

论文配图:AHPA: Adaptive Hierarchical Prior Alignment for Diffusion Transformers
图 1 · 摘自论文原文
  • 根据噪声水平自适应选择多层级先验特征进行对齐
  • 在高噪声阶段侧重语义布局,低噪声阶段强调细节结构
  • 无需外部编码器,轻量设计且推理无额外开销

表示对齐已成为加速扩散变换器训练的有效范式。然而,现有方法通常在整个去噪轨迹中采用固定监督目标或固定对齐粒度,无论指导信号来自外部视觉编码器、内部自表示还是VAE提取特征。我们指出,这种与时间步无关的对齐方式次优,因为表示监督的有效粒度随信噪比系统性变化:高噪声阶段更需粗粒度语义与布局锚定,低噪声阶段则应强调空间细节和结构忠实性。静态单层级监督导致表征错配。为此,我们提出自适应分层先验对齐(AHPA),利用冻结VAE编码器固有的分层表示,不只使用单一压缩潜在变量作为对齐目标,而是提取从局部几何、空间拓扑到粗粒度语义布局的多层级特征,提供互补先验。时间步条件的动态路由模块自适应选择并加权这些分层先验,使对齐粒度与模型演化需求同步。大量实验表明,AHPA在收敛速度和生成质量上优于基线,训练中无需外部编码器监督,且推理时无额外开销。

原文摘要 · Abstract (English)

Representation alignment has recently emerged as an effective paradigm for accelerating Diffusion Transformer training. Despite their success, existing alignment methods typically impose a fixed supervision target or a fixed alignment granularity throughout the entire denoising trajectory, whether the guidance is provided by external vision encoders, internal self-representations, or VAE-derived features. We argue that such timestep-agnostic alignment is suboptimal because the useful granularity of representation supervision changes systematically with the signal-to-noise ratio. In high-noise regimes, diffusion models benefit more from coarse semantic and layout-level anchoring, whereas in low-noise regimes, the training signal should emphasize spatially detailed and structurally faithful refinement. This non-stationary alignment behavior creates a representational mismatch for static single-level supervisors. To address this issue, we propose Adaptive Hierarchical Prior Alignment (AHPA), a lightweight alignment framework that exploits the hierarchical representations naturally embedded in the frozen VAE encoder. Instead of using only a single compressed latent as the alignment target, AHPA extracts multi-level VAE features that provide complementary priors ranging from local geometry and spatial topology to coarse semantic layout. A timestep-conditioned Dynamic Router adaptively selects and weights these hierarchical priors along the denoising trajectory, thereby synchronizing the alignment granularity with the model's evolving training needs. Extensive experiments show that AHPA improves convergence and generation quality over baselines and incurs no additional inference cost while avoiding external encoder supervision during training.

扩散模型自适应对齐分层先验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。