解决电商图像生成中因过度补全导致商品特征失真的问题
E-Commerce Inpainting with Mask Guidance in Controlnet for Reducing Overcompletion
- 用实例掩码微调的修复模型减少过度假设
- 无需训练的掩码引导策略，约束生成过程保持商品原貌
- 适合电商图像修复与可控生成场景
电商图像生成是电商领域的重要需求，目标是恢复与主商品匹配的缺失背景。后AIGC时代，扩散模型被广泛用于生成商品图像，取得显著成果。本文系统分析并解决扩散模型生成中的核心痛点——过度假设，即难以保持商品特征。提出两种解决方案：1. 使用实例掩码微调的修复模型缓解该现象；2. 采用无需训练的掩码引导方法，在结合ControlNet与UNet生成主商品时，将优化后的商品掩码作为约束，避免对商品本身的过度补全。该方法在实际应用中表现优异，有望为该领域提供有价值的参考。
原文摘要 · Abstract (English)
E-commerce image generation has always been one of the core demands in the e-commerce field. The goal is to restore the missing background that matches the main product given. In the post-AIGC era, diffusion models are primarily used to generate product images, achieving impressive results. This paper systematically analyzes and addresses a core pain point in diffusion model generation: overcompletion, which refers to the difficulty in maintaining product features. We propose two solutions: 1. Using an instance mask fine-tuned inpainting model to mitigate this phenomenon; 2. Adopting a train-free mask guidance approach, which incorporates refined product masks as constraints when combining ControlNet and UNet to generate the main product, thereby avoiding overcompletion of the product. Our method has achieved promising results in practical applications and we hope it can serve as an inspiring technical report in this field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。