无需训练即可精准编辑图像,保持背景真实且语义一致。
VENUS: Visual Editing with Noise Inversion Using Scene Graphs

- 用场景图分离目标与背景,结合噪声反演保持未编辑区域细节。
- 在PIE-Bench上提升PSNR至24.80、SSIM达0.84,LPIPS降至0.070。
- 运行时间从分钟级缩短至30秒内,适合快速高保真图像编辑。
当前基于文本的图像编辑模型难以兼顾背景保留与语义一致性,常生成全新图像或无法实现预期修改。场景图编辑虽能提升可控性,但依赖模型微调,计算成本高。为此,我们提出VENUS(Visual Editing with Noise Inversion Using Scene Graphs),一种无需训练的场景图引导图像编辑框架。VENUS采用分离提示条件策略,将编辑目标与其背景解耦,同时利用噪声反演保留未编辑区域的真实感。该方法融合多模态大语言模型提取的场景图与扩散模型骨干网络,无需额外训练。实验表明,VENUS在PIE-Bench上将PSNR从22.45提升至24.80,SSIM从0.79增至0.84,LPIPS由0.100降至0.070;相比SGEdit,CLIP相似度提升至24.97。在EditVal上,其DINO得分达0.87,单图推理时间从6-10分钟压缩至20-30秒。此外,VENUS在文本编辑基线如LEdit++和P2P+DirInv上也表现更优,跨范式持续领先。
原文摘要 · Abstract (English)
State-of-the-art text-based image editing models often struggle to balance background preservation with semantic consistency, frequently resulting either in the synthesis of entirely new images or in outputs that fail to realize the intended edits. In contrast, scene graph-based image editing addresses this limitation by providing a structured representation of semantic entities and their relations, thereby offering improved controllability. However, existing scene graph editing methods typically depend on model fine-tuning, which incurs high computational cost and limits scalability. To this end, we introduce VENUS (Visual Editing with Noise inversion Using Scene graphs), a training-free framework for scene graph-guided image editing. Specifically, VENUS employs a split prompt conditioning strategy that disentangles the target object of the edit from its background context, while simultaneously leveraging noise inversion to preserve fidelity in unedited regions. Moreover, our proposed approach integrates scene graphs extracted from multimodal large language models with diffusion backbones, without requiring any additional training. Empirically, VENUS substantially improves both background preservation and semantic alignment on PIE-Bench, increasing PSNR from 22.45 to 24.80, SSIM from 0.79 to 0.84, and reducing LPIPS from 0.100 to 0.070 relative to the state-of-the-art scene graph editing model (SGEdit). In addition, VENUS enhances semantic consistency as measured by CLIP similarity (24.97 vs. 24.19). On EditVal, VENUS achieves the highest fidelity with a 0.87 DINO score and, crucially, reduces per-image runtime from 6-10 minutes to only 20-30 seconds. Beyond scene graph-based editing, VENUS also surpasses strong text-based editing baselines such as LEDIT++ and P2P+DirInv, thereby demonstrating consistent improvements across both paradigms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。