让图像物体编辑既准又真,解决3D几何与背景合成的矛盾。
GeoEdit: Geometry-Aware Object Editing via Dual-Branch Denoising

- 分两路去噪:物体保形,背景自由生成。
- 在噪声方差匹配时注入3D约束,避免伪影和模糊。
- 适合需要精确几何编辑的视觉设计、虚拟拍摄场景。
基于扩散模型的图像编辑在单图中精准操控物体(平移、旋转、缩放)并遵循三维物理约束仍属难题。现有2D方法缺乏空间感知,易产生透视错误;强制结构代理进入隐空间会破坏方差同质性,导致自注意力泄露,引发鬼影和背景模糊。核心矛盾在于:移动物体需保持刚性几何,未覆盖背景却需自由合成合理内容。本文提出GeoEdit,一种无需训练的Lift-Manipulate-Render-Denoise流程。通过3D解耦场景与物体,利用点对应对齐,生成带有结构深度图的几何对齐代理。双分支去噪阶段中,视频扩散主干保留物体身份,3D约束在匹配噪声方差的窄去噪窗口内注入前景(方差同质注入),背景则自由去噪。因注入信号匹配原生隐空间统计,自注意力不受扰动。我们还构建了GeoEditBench,一个包含物体平移、旋转与相机移动的姿势感知基准,配备姿势感知评估指标。实验表明,几何精度、身份保真度与背景质量均显著提升。代码已开源:https://github.com/Heey731/GeoEdit。
原文摘要 · Abstract (English)
Precisely manipulating objects in a single photograph (translation, rotation, scaling) while obeying 3D physical constraints remains unsolved for diffusion-based editors. Current 2D methods lack spatial awareness and produce perspective violations. Forcing structural proxies into the latent space also disrupts variance homogeneity, and the resulting self-attention leakage leads to ghosting and background blur. The core difficulty is asymmetric: the relocated object must follow a rigid geometry, yet the uncovered background needs freedom to synthesize plausible content. We present GeoEdit, a training-free Lift-Manipulate-Render-Denoise pipeline that satisfies both constraints. We decouple scene and object in 3D, align them through point correspondence, and render a geometry-aligned proxy with a structural depth map. A Dual-Branch Denoising stage then refines this proxy: a video diffusion backbone preserves object identity, while 3D constraints are injected into the foreground within a narrow denoising window at matching noise variance (variance-homogeneous injection). The background denoises freely. Because the injected signal matches the native latent statistics, self-attention stays undisturbed. We also introduce GeoEditBench, a pose-aware benchmark covering object translation, object rotation, and camera movement with pose-aware evaluation metrics. Experiments confirm consistent gains in geometric accuracy, identity fidelity, and background quality. Our codes are available at https://github.com/Heey731/GeoEdit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。