arXiv:2501.15043cs.CV2025-01IJCAI被引 8

用户用提示词指定要消除的阴影对象,无需标注即可精准去影。

Prompt-Aware Controllable Shadow Removal

  • 通过用户提示生成目标阴影掩码,实现针对性去影
  • 新模型在多个数据集上优于现有方法,去影更精准
  • 适合需要精细控制阴影去除场景的开发者和研究者

阴影去除旨在恢复阴影区域中的图像内容。尽管基于深度学习的方法已取得良好效果,但仍面临两大挑战:一是无差别地移除所有阴影,二是可控去除依赖精确的阴影区域掩码。为此,本文提出一种新型范式:提示感知的可控阴影去除。该范式允许用户通过提示(如点、线或主体掩码)指定需去除阴影的具体对象,无需阴影标注即可实现灵活、可控制的去影。我们提出一个端到端可学习模型——提示感知可控阴影去除网络(PACSRNet),包含两个核心模块:提示感知模块根据用户提示生成目标阴影掩码;阴影去除模块利用该掩码及前一模块提供的特征信息,恢复阴影区域内容。为提升性能,我们在阴影去除模块中引入线性融合机制,实现提示引导。此外,针对现有数据集缺乏多样化用户提示的问题,我们构建了一个专为提示式可控阴影去除设计的新数据集。大量实验表明,PACSRNet在多个基准上均表现出色,显著优于现有方法。

原文摘要 · Abstract (English)

Shadow removal aims to restore the image content in shadowed regions. While deep learning-based methods have shown promising results, they still face key challenges: 1) uncontrolled removal of all shadows, or 2) controllable removal but heavily relies on precise shadow region masks. To address these issues, we introduce a novel paradigm: prompt-aware controllable shadow removal. Unlike existing approaches, our paradigm allows for targeted shadow removal from specific subjects based on user prompts (e.g., dots, lines, or subject masks). This approach eliminates the need for shadow annotations and offers flexible, user-controlled shadow removal. Specifically, we propose an end-to-end learnable model, the Prompt-Aware Controllable Shadow Removal Network (PACSRNet). PACSRNet consists of two key modules: a prompt-aware module that generates shadow masks for the specified subject based on the user prompt, and a shadow removal module that uses the shadow prior from the first module to restore the content in the shadowed regions. Additionally, we enhance the shadow removal module by incorporating feature information from the prompt-aware module through a linear operation, providing prompt-guided support for shadow removal. Recognizing that existing shadow removal datasets lack diverse user prompts, we contribute a new dataset specifically designed for prompt-based controllable shadow removal. Extensive experimental results demonstrate the effectiveness and superiority of PACSRNet.

阴影去除提示控制图像修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。