通过定义操作区域和类型,解决拖拽编辑的模糊性问题。
DragNeXt: Rethinking Drag-Based Image Editing
- 将拖拽编辑重定义为区域形变、旋转与平移,明确用户操作意图。
- 在NextBench上显著优于现有方法,生成质量更高且过程更简洁。
- 适合需要精准控制图像编辑的设计师与研究人员。
基于拖拽的图像编辑(DBIE)允许用户通过直接拖动图像中的对象来实现操控,近期受到广泛关注。然而,该技术面临两大挑战:(i)点级拖拽常存在高度歧义,难以对齐用户意图;(ii)现有方法依赖运动监督与点追踪交替进行,流程繁琐且难以生成高质量结果。为此,本文从新视角重新思考DBIE——将其定义为用户指定操控区域的形变、旋转与平移。通过显式指定拖拽区域和操作类型,有效缓解歧义问题。进一步提出简单而高效的编辑框架DragNeXt,将DBIE统一建模为潜在区域优化(LRO)问题,并通过渐进式反向自干预(PBSI)求解,简化流程的同时充分挖掘区域级结构信息与中间状态的渐进引导。在自建的NextBench数据集上验证,实验表明该方法显著超越现有方法。代码将开源于GitHub。
原文摘要 · Abstract (English)
Drag-Based Image Editing (DBIE), which allows users to manipulate images by directly dragging objects within them, has recently attracted much attention from the community. However, it faces two key challenges: (\emph{\textcolor{magenta}{i}}) point-based drag is often highly ambiguous and difficult to align with users' intentions; (\emph{\textcolor{magenta}{ii}}) current DBIE methods primarily rely on alternating between motion supervision and point tracking, which is not only cumbersome but also fails to produce high-quality results. These limitations motivate us to explore DBIE from a new perspective -- redefining it as deformation, rotation, and translation of user-specified handle regions. Thereby, by requiring users to explicitly specify both drag areas and types, we can effectively address the ambiguity issue. Furthermore, we propose a simple-yet-effective editing framework, dubbed \textcolor{SkyBlue}{\textbf{DragNeXt}}. It unifies DBIE as a Latent Region Optimization (LRO) problem and solves it through Progressive Backward Self-Intervention (PBSI), simplifying the overall procedure of DBIE while further enhancing quality by fully leveraging region-level structure information and progressive guidance from intermediate drag states. We validate \textcolor{SkyBlue}{\textbf{DragNeXt}} on our NextBench, and extensive experiments demonstrate that our proposed method can significantly outperform existing approaches. Code will be released on github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。