将位置编码扩展为3D场，提升图像生成的几何建模能力
Positional Encoding Field
- 把2D位置编码升级为含深度与分层结构的3D位置场
- 在单图新视角生成任务上达到当前最佳性能
- 适合需要精确空间控制的图像生成与编辑场景
扩散Transformer（DiTs）已成为视觉生成的主流架构，通过将图像表示为带位置编码（PEs）的块令牌，结合了Transformer的可扩展性与空间/时间先验。本文重新审视DiTs对视觉内容的组织方式，发现块令牌具有显著独立性：即使位置编码被扰动，模型仍能生成全局连贯输出，表明空间一致性主要由位置编码决定。基于此，我们提出位置编码场（PE-Field），将位置编码从2D平面扩展至结构化3D场，引入深度感知编码用于体素推理，并采用分层编码实现细粒度子块控制，使DiTs能够直接在3D空间建模几何。使用PE-Field增强的DiT在单图新视角生成任务上达到最新水平，并可泛化至可控空间图像编辑。
原文摘要 · Abstract (English)
Diffusion Transformers (DiTs) have emerged as the dominant architecture for visual generation, powering state-of-the-art image and video models. By representing images as patch tokens with positional encodings (PEs), DiTs combine Transformer scalability with spatial and temporal inductive biases. In this work, we revisit how DiTs organize visual content and discover that patch tokens exhibit a surprising degree of independence: even when PEs are perturbed, DiTs still produce globally coherent outputs, indicating that spatial coherence is primarily governed by PEs. Motivated by this finding, we introduce the Positional Encoding Field (PE-Field), which extends positional encodings from the 2D plane to a structured 3D field. PE-Field incorporates depth-aware encodings for volumetric reasoning and hierarchical encodings for fine-grained sub-patch control, enabling DiTs to model geometry directly in 3D space. Our PE-Field-augmented DiT achieves state-of-the-art performance on single-image novel view synthesis and generalizes to controllable spatial image editing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。