arXiv:2606.05142cs.CVcs.AI2026-06被引 1

无需训练即可实现复杂非刚性场景编辑,保持多视角一致性。

GeM-NR: Geometry-Aware Multi-View Editing for Nonrigid Scene Changes

论文配图:GeM-NR: Geometry-Aware Multi-View Editing for Nonrigid Scene Changes
图 1 · 摘自论文原文
  • 通过优化3D点云对齐实现几何感知的多视角编辑
  • 支持大幅改变场景结构与外观的任意编辑任务
  • 适合需要灵活3D内容生成的设计师和开发者

基于生成模型的多视角图像编辑近年推动了通用3D内容生成与定制的发展。现有方法主要聚焦于刚性或仅外观修改,依赖未编辑场景的几何结构,难以处理结构变化大的编辑。当前非刚性方法仅限于物体移除与插入,受限于训练数据。本文提出GeM-NR,一种快速、灵活的免训练方法,可实现包括大幅改变场景几何与外观在内的通用多视角一致编辑。给定一个经2D编辑器处理的锚点图像及一个未编辑的查询图像,GeM-NR使查询图像在多视角上一致地匹配锚点编辑。方法包含三个阶段:(i) 深度图估计,提出策略以最大化编辑后与未编辑场景3D点云的对齐;(ii) 投影至查询视角;(iii) 基于未编辑查询图像对结果进行精细化调整。实验表明,该方法能有效处理几何与外观显著变化的编辑任务。定量与定性评估显示,GeM-NR在多种编辑任务中均实现了领先的编辑质量,以及跨多视角的几何与光照一致性表现。

原文摘要 · Abstract (English)

Recent developments in multi-view image editing with generative models have brought us a step closer toward general 3D content generation and customization. Most existing works focus on rigid or appearance-only edits by utilizing the geometry of the unedited scene. This naturally limits these methods to edits that preserve the underlying scene structure. Current nonrigid approaches are limited to object removal and insertion, reflecting the data they are trained on. General nonrigid edits, i.e., edits that substantially and arbitrarily change the scene geometry, remain challenging for existing methods. We propose GeM-NR, a fast and flexible training-free approach for general multi-view consistent image editing, including edits that drastically change the geometry and appearance of the scene. Given an anchor image edited with a chosen 2D editor and a query unedited image, GeM-NR edits the query image consistently with the anchor edit. The method incorporates multiple stages: (i) depth map estimation, where we propose a strategy to maximize the alignment between the 3D point clouds of the edited and unedited scenes, (ii) projection onto a query viewpoint, and (iii) refinement of the obtained image conditioned on the unedited query. We demonstrate the ability of our method to handle edits with significant changes in geometry and appearance, something that existing methods struggle with. We perform an extensive evaluation showing that GeM-NR improves consistency for a wide variety of edit tasks, including generating 3D representations of the edited scene. Both quantitative and qualitative results indicate the state-of-the-art performance of our method in terms of edit quality as well as geometric and photometric consistency across multiple views.

多视角编辑非刚性编辑3D生成几何感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。