提出HyperDiT，解决像素级扩散模型的细节与语义矛盾问题。
HyperDiT: Hyper-Connected Transformers for High-Fidelity Pixel-Space Diffusion
- 用跨尺度交叉注意力连接细粒度与高层语义
- 在ImageNet上实现1.56的FID新纪录
- 适合需要高保真图像生成的研究者
像素空间扩散模型避免了变分自编码器的重构瓶颈，但面临根本性“粒度困境”：捕捉全局语义需大块尺度，生成高保真细节则依赖细粒度输入。为此，我们提出HyperDiT，通过建立超连通跨尺度交互机制，弥合语义与像素流形之间的鸿沟。不同于采用AdaLN注入语义，HyperDiT使用交叉注意力机制，使细粒度令牌可全局查询多层级语义锚点。为解决多尺度交互中的空间错配问题，引入尺度感知旋转位置编码（SA-RoPE），确保不同块大小令牌间的精确几何对齐。此外，通过引入寄存器学习预训练视觉基础模型（VFM）的密集语义，有效减少生成幻觉与伪影。大量实验表明，HyperDiT在直接像素空间中实现了ImageNet $256 imes256$ 上的最新最佳FID分数1.56。结合细粒度流与语义引导，HyperDiT提供了一种高保真像素生成的优越范式。
原文摘要 · Abstract (English)
Pixel-space diffusion models bypass the reconstruction bottleneck of Variational Autoencoders (VAEs) but face a fundamental "granularity dilemma": capturing global semantics favors large patch scales, while generating high-fidelity details demands fine-grained inputs. To address this issue, we propose HyperDiT, a unified framework establishing Hyper-Connected Cross-Scale Interactions to bridge the semantic and pixel manifold. Diverging from injecting semantics by AdaLN, HyperDiT utilizes Cross-Attention mechanisms, enabling fine-grained tokens to query multi-level semantic anchors globally. To resolve the spatial mismatch during multi-scale interactions, we introduce Scale-Aware Rotary Position Embedding (SA-RoPE) to ensure precise geometric alignment among tokens of varying patch sizes. Furthermore, we incorporate Registers to learn the dense semantics from a pretrained Visual Foundation Model (VFM), effectively reducing generation hallucination and artifacts. Extensive experiments demonstrate that HyperDiT achieves state-of-the-art (SoTA) FID of $\mathbf{1.56}$ on ImageNet $256\times256$ directly within the pixel space. By combining the fine-grained stream with semantic guidance, HyperDiT offers a superior paradigm for high-fidelity pixel generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。