一通搞定多种3D编辑,速度从几十分钟缩至两分钟。
Omni-3DEdit: Generalized Versatile 3D Editing in One-Pass
- 用学习方法统一处理各类3D编辑,避免逐任务设计规则。
- 单次前向传播完成编辑,推理时间从数分钟降至约2分钟。
- 适用于需要快速生成多样3D内容的设计师与开发者。
现有指令驱动的3D编辑方法多依赖2D模型引导显式、迭代优化3D表示,存在两大缺陷:一是缺乏统一设计,因3D几何显式操作需依赖任务特定规则(如外观编辑需保留原始几何,移除则需修改几何);二是迭代优化耗时极长,常需数千次2D/3D更新调用。本文提出Omni-3DEdit,一个统一的、基于学习的模型,隐式泛化多种3D编辑任务。为解决训练数据稀缺问题,我们构建数据流水线,合成大量高质量的多视角配对编辑样本。随后,将预训练生成模型SEVA作为骨干,通过序列空间拼接源视图潜在表示与条件标记。引入双流LoRA模块,解耦不同视角线索,显著增强模型表征能力。作为学习型模型,它无需耗时的在线优化,可在一次前向传播中完成各类3D编辑,推理时间由数十分钟降至约两分钟。大量实验验证了Omni-3DEdit的有效性与高效性。
原文摘要 · Abstract (English)
Most instruction-driven 3D editing methods rely on 2D models to guide the explicit and iterative optimization of 3D representations. This paradigm, however, suffers from two primary drawbacks. First, it lacks a universal design of different 3D editing tasks because the explicit manipulation of 3D geometry necessitates task-dependent rules, e.g., 3D appearance editing demands inherent source 3D geometry, while 3D removal alters source geometry. Second, the iterative optimization process is highly time-consuming, often requiring thousands of invocations of 2D/3D updating. We present Omni-3DEdit, a unified, learning-based model that generalizes various 3D editing tasks implicitly. One key challenge to achieve our goal is the scarcity of paired source-edited multi-view assets for training. To address this issue, we construct a data pipeline, synthesizing a relatively rich number of high-quality paired multi-view editing samples. Subsequently, we adapt the pre-trained generative model SEVA as our backbone by concatenating source view latents along with conditional tokens in sequence space. A dual-stream LoRA module is proposed to disentangle different view cues, largely enhancing our model's representational learning capability. As a learning-based model, our model is free of the time-consuming online optimization, and it can complete various 3D editing tasks in one forward pass, reducing the inference time from tens of minutes to approximately two minutes. Extensive experiments demonstrate the effectiveness and efficiency of Omni-3DEdit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。