用样例对指导人脸精修,实现细微结构调整与身份保持。
MirrorPPR: Exemplar-Based Portrait Photo Retouching

- 通过样例对提取微调操作,注入扩散Transformer模型进行精修。
- 在4700万级数据集上训练,显著提升修图质量与身份一致性。
- 适合需要精准人脸修饰的设计师和图像编辑从业者。
尽管文本引导图像编辑进展显著,但在结构化人像精修方面仍受限。文本难以描述面部特征与体态比例的细微变化。为此,我们提出基于样例的人像精修方法,给定一对样例图像,模型需推断并应用相同的精修操作于新查询图像。现有样例方法多针对明显视觉变化,而人像精修涉及极细微且局部化的修改,准确提取与迁移难度大。为此,我们提出MirrorPPR框架,使用精修操作提取器捕捉样例对间的细微差异,并通过连接模块与低秩适配(LoRA)注入预训练扩散Transformer(DiT)。此外,跨身份训练对的严格对齐因操作错位而受阻,我们提出先进自增强策略确保操作严格对齐。为缓解数据稀缺问题,我们构建了超过4700万对精修图像的数据集MirrorPPR47M,分为模拟与专业子集,支持渐进式课程学习以优化网络。大量实验表明,MirrorPPR在精修质量与身份保留上均显著优于现有基线。
原文摘要 · Abstract (English)
While text-guided image editing has made remarkable progress, it remains limited in structural portrait retouching. Textual descriptions struggle to convey fine-grained changes to facial features and body proportions. To address this gap, we introduce Exemplar-Based Portrait Photo Retouching, where the model is given an exemplar pair and tasked with inferring and applying the same retouching operations to a new query image. Existing exemplar-based editing methods primarily focus on tasks with pronounced visual transformations. In contrast, structural portrait retouching involves extremely delicate and localized modifications, making accurate extraction and transfer of these edits challenging. To tackle this, we propose MirrorPPR, a novel framework designed to capture and transfer subtle structural retouching operations. Our method uses a Retouching Operation Extractor to capture the subtle differences from the exemplar pair. The extracted representations are then injected into a pre-trained Diffusion Transformer (DiT) through a connector and Low-Rank Adaptation (LoRA) modules. Furthermore, constructing perfectly aligned cross-identity training pairs is severely hindered by operation misalignment. To overcome this, we propose an advanced data self-augmentation paradigm that ensures strictly aligned retouching operations. To alleviate data scarcity and support this novel task, we introduce MirrorPPR47M, a large-scale dataset with over 47 million retouched pairs. By structuring the dataset into simulated and professional subsets, we enable progressive curriculum learning to smoothly optimize the network. Extensive experiments demonstrate that MirrorPPR significantly outperforms existing baselines in both retouching quality and identity preservation. The project page is available at https://sjtu-deng-lab.github.io/MirrorPPR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。