arXiv:2604.06870cs.CV2026-04被引 7

专注局部细节修复,精准改图不伤背景。

RefineAnything: Multimodal Region-Specific Refinement for Perfect Local Details

  • 用聚焦重采样策略把分辨率重点给目标区域,提升细节还原力。
  • 在新构建的RefineEval基准上,背景保持近乎完美,优于现有方法。
  • 支持有参考和无参考两种模式,适合需要高精度修图的场景。

我们提出区域特定图像精修这一新问题:给定一张输入图像和用户指定的区域(如涂抹掩码或边界框),目标是在不改变任何未编辑像素的前提下恢复细粒度细节。尽管图像生成技术快速发展,现代模型仍普遍存在局部细节丢失问题(如文字、标志、细小结构扭曲)。现有指令驱动编辑模型侧重粗粒度语义修改,常忽略细微缺陷,或在目标区域较小时意外改动背景。我们提出RefineAnything,一种基于扩散模型的多模态精修方法,支持有参考和无参考两种模式。基于一个反直觉观察——在固定VAE输入分辨率下,裁剪与重缩放可显著提升局部重建效果——我们设计了焦点-精修策略:将分辨率预算集中于目标区域,再通过融合掩码贴回确保背景严格不变。进一步引入边界感知一致性损失,减少拼接痕迹,提升贴回自然度。为支持该任务,我们构建了Refine-30K数据集(20K有参考,10K无参考),并提出RefineEval基准,评估编辑区域保真度与背景一致性。在RefineEval上,RefineAnything显著优于基线,实现近乎完美的背景保留,为高精度局部精修提供实用解决方案。

原文摘要 · Abstract (English)

We introduce region-specific image refinement as a dedicated problem setting: given an input image and a user-specified region (e.g., a scribble mask or a bounding box), the goal is to restore fine-grained details while keeping all non-edited pixels strictly unchanged. Despite rapid progress in image generation, modern models still frequently suffer from local detail collapse (e.g., distorted text, logos, and thin structures). Existing instruction-driven editing models emphasize coarse-grained semantic edits and often either overlook subtle local defects or inadvertently change the background, especially when the region of interest occupies only a small portion of a fixed-resolution input. We present RefineAnything, a multimodal diffusion-based refinement model that supports both reference-based and reference-free refinement. Building on a counter-intuitive observation that crop-and-resize can substantially improve local reconstruction under a fixed VAE input resolution, we propose Focus-and-Refine, a region-focused refinement-and-paste-back strategy that improves refinement effectiveness and efficiency by reallocating the resolution budget to the target region, while a blended-mask paste-back guarantees strict background preservation. We further introduce a boundary-aware Boundary Consistency Loss to reduce seam artifacts and improve paste-back naturalness. To support this new setting, we construct Refine-30K (20K reference-based and 10K reference-free samples) and introduce RefineEval, a benchmark that evaluates both edited-region fidelity and background consistency. On RefineEval, RefineAnything achieves strong improvements over competitive baselines and near-perfect background preservation, establishing a practical solution for high-precision local refinement. Project Page: https://limuloo.github.io/RefineAnything/.

图像修复扩散模型局部精修

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。