根据编辑指令自动调整推理深度和区域,提升图像编辑效率与准确性。
PhysEdit: Physically-Consistent Region-Aware Image Editing via Adaptive Spatio-Temporal Reasoning

- 按指令复杂度动态分配推理步数和区域,实现自适应计算。
- 在737个案例中提速1.18倍,外观编辑最快达1.52倍。
- 适合需要高效精准编辑的视觉生成与内容创作场景。
图像编辑指令类型多样:颜色替换、物体插入与物理动作修改对空间覆盖和推理深度要求不同,但现有基于推理的编辑器对所有指令使用固定推理流程。本文提出PhysEdit框架,主张在空间与时间维度上实现自适应性。其核心包含两个无需重训练主干网络的推理时模块:(1) 复杂度自适应推理深度(CARD),从指令与参考图直接预测编辑复杂度,并动态分配推理步数N_r与推理标记长度r,将固定推理流程转为条件计算问题;(2) 空间推理掩码(SRM),通过交叉注意力提取指令相关的空间先验,将推理限制在语义需干预的区域。在包含737个案例的ImgEdit Basic-Edit Suite上,PhysEdit相较强基线实现1.18倍的墙钟速度提升(64.3秒 vs. 76.1秒/样本),同时轻微提升指令遵循度(CLIP-T 0.2283 vs. 0.2266,+0.7%),身份保持与噪声水平相当(CLIP-I 0.8246 vs. 0.8280)。速度提升具类别依赖性,外观级编辑最高达1.52倍,验证了CARD的自适应分配是主要效率来源。30样本小规模消融实验明确各模块贡献。
原文摘要 · Abstract (English)
Image editing instructions are heterogeneous: a color swap, an object insertion, and a physical-action edit all demand different spatial coverage and different reasoning depth, yet existing reasoning-based editors apply a single fixed inference recipe to every instruction. We argue that adaptivity along both the spatial and temporal axes is the missing degree of freedom, and we present PhysEdit, an editing framework built around this principle. PhysEdit introduces two inference-time modules that compose without retraining the backbone. At its core, (1) Complexity-Adaptive Reasoning Depth (CARD) predicts edit complexity directly from the instruction and reference image and allocates the reasoning step count N_r and reasoning-token length r per sample -- turning a previously fixed inference schedule into a conditional-computation problem. CARD is supported by (2) a Spatial Reasoning Mask (SRM) that extracts an instruction-conditioned spatial prior from cross-attention to confine reasoning to regions that semantically require it. On the full 737-case ImgEdit Basic-Edit Suite, PhysEdit delivers a 1.18x wall-clock speedup (64.3s vs. 76.1s per sample) over a strong reasoning baseline while slightly improving instruction adherence (CLIP-T 0.2283 vs. 0.2266, +0.7%) and matching identity preservation within noise (CLIP-I 0.8246 vs. 0.8280). The speedup is category-dependent and reaches 1.52x on appearance-level edits, validating CARD's adaptive allocation as the principal source of efficiency gain. A 30-sample pilot with full ablations isolates the contribution of each module.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。