arXiv:2504.14606cs.CV2025-04ICLR被引 1

提出3D与实例感知的图像抠图框架,解决复杂边界下多人重叠难题。

MP-Mat: A 3D-and-Instance-Aware Human Matting and Editing Framework with Multiplane Representation

论文配图:MP-Mat: A 3D-and-Instance-Aware Human Matting and Editing Framework with Multiplane Representation
图 1 · 摘自论文原文
  • 用多平面表示从几何和实例两个层面分解场景,提升3D感知与边界精度。
  • 在多个数据集上实现最优抠图效果,尤其在遮挡区域表现显著优于现有方法。
  • 支持零样本高效图像编辑,适合需要精准抠图与后期处理的应用场景。

人体实例抠图旨在为图像中每个体人物实例估计透明度蒙版,其挑战在于复杂情况下需分离属于多个实例的交错像素,尤其是在毛发与细小边界结构处。本文提出MP-Mat,一种基于多平面表示的3D与实例感知抠图框架,从场景几何与实例两个视角设计多平面表示:首先构建特征级多平面表示,依据深度差异将场景分割为多个平面,使表征具备3D感知能力,有效区分不同空间位置的实例,增强遮挡区域的可解释性与边界处理能力;其次引入实例级多平面表示,对每个实例(包括背景作为特殊实例)分别建模透明度与颜色信息,促进前景与背景内容感知,便于下游任务如图像编辑;该表示可复用以实现高效可控的实例级图像编辑。大量实验验证了MP-Mat在抠图任务中的明显优势,且在图像编辑任务中表现卓越,甚至在零样本推理下超越训练过的专用编辑模型。代码已开源。

原文摘要 · Abstract (English)

Human instance matting aims to estimate an alpha matte for each human instance in an image, which is challenging as it easily fails in complex cases requiring disentangling mingled pixels belonging to multiple instances along hairy and thin boundary structures. In this work, we address this by introducing MP-Mat, a novel 3D-and-instance-aware matting framework with multiplane representation, where the multiplane concept is designed from two different perspectives: scene geometry level and instance level. Specifically, we first build feature-level multiplane representations to split the scene into multiple planes based on depth differences. This approach makes the scene representation 3D-aware, and can serve as an effective clue for splitting instances in different 3D positions, thereby improving interpretability and boundary handling ability especially in occlusion areas. Then, we introduce another multiplane representation that splits the scene in an instance-level perspective, and represents each instance with both matte and color. We also treat background as a special instance, which is often overlooked by existing methods. Such an instance-level representation facilitates both foreground and background content awareness, and is useful for other down-stream tasks like image editing. Once built, the representation can be reused to realize controllable instance-level image editing with high efficiency. Extensive experiments validate the clear advantage of MP-Mat in matting task. We also demonstrate its superiority in image editing tasks, an area under-explored by existing matting-focused methods, where our approach under zero-shot inference even outperforms trained specialized image editing techniques by large margins. Code is open-sourced at https://github.com/JiaoSiyi/MPMat.git}.

图像抠图3D感知实例分割图像编辑

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。