arXiv:2506.07738cs.CV2025-06International Conf…被引 3

用扩散模型从图像中提取标准化设计资产,解决复杂场景下的精准提取难题。

AssetDropper: Asset Extraction via Diffusion Models with Reward-Driven Optimization

  • 基于扩散模型和奖励驱动优化,实现从参考图中提取标准视图资产。
  • 在20万+合成数据与数千真实样本上验证,效果领先现有方法。
  • 适合需要高效获取可复用设计素材的艺术家与设计师使用。

生成模型研究多聚焦于生成成品视觉内容,但设计师更需要标准化的资产库,这一领域尚未被生成技术显著提升。尽管开放世界图像蕴含丰富素材,但高效提取高质量、标准化资产仍具挑战。为此,我们提出AssetDropper,首个从参考图像中提取资产的框架,为艺术家提供开放世界的资产调色板。该模型能准确提取选定主体的正面视图,有效处理透视扭曲与遮挡等复杂情况。我们构建了超过20万张图像-主体对的合成数据集,并建立包含数千样本的真实世界基准用于评估,推动下游任务研究。为确保提取结果与提示语一致,引入预训练奖励模型实现闭环反馈:该模型执行逆向任务——将提取资产回贴至原图,以增强一致性并减少幻觉。大量实验表明,借助奖励驱动优化,AssetDropper在资产提取任务中达到当前最优性能。

原文摘要 · Abstract (English)

Recent research on generative models has primarily focused on creating product-ready visual outputs; however, designers often favor access to standardized asset libraries, a domain that has yet to be significantly enhanced by generative capabilities. Although open-world scenes provide ample raw materials for designers, efficiently extracting high-quality, standardized assets remains a challenge. To address this, we introduce AssetDropper, the first framework designed to extract assets from reference images, providing artists with an open-world asset palette. Our model adeptly extracts a front view of selected subjects from input images, effectively handling complex scenarios such as perspective distortion and subject occlusion. We establish a synthetic dataset of more than 200,000 image-subject pairs and a real-world benchmark with thousands more for evaluation, facilitating the exploration of future research in downstream tasks. Furthermore, to ensure precise asset extraction that aligns well with the image prompts, we employ a pre-trained reward model to fulfill a closed-loop with feedback. We design the reward model to perform an inverse task that pastes the extracted assets back into the reference sources, which assists training with additional consistency and mitigates hallucination. Extensive experiments show that, with the aid of reward-driven optimization, AssetDropper achieves the state-of-the-art results in asset extraction. Project page: AssetDropper.github.io.

扩散模型图像提取设计工具奖励机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。