提升视觉推理分割的分辨率感知能力,实现更精准的细粒度分割。
HRSeg: High-Resolution Visual Perception and Enhancement for Reasoning Segmentation
- 采用分块处理高分辨率图像,融合多粒度局部与全局特征。
- 通过高分辨率信息增强掩码特征,提升与文本特征对齐精度。
- 适合需要精细语义理解的复杂图像分割任务,如开放世界场景。
推理分割任务需根据隐含用户指令分割图像中的物体,涉及上下文线索和开放世界知识等细微信息。现有方法受限于低分辨率感知能力,因视觉编码器通常在低分辨率下预训练。简单地插值视觉编码器的位置嵌入以提升分辨率,仅带来微弱性能改善且计算开销巨大。为此,我们提出HRSeg,一种具备高分辨率细粒度感知能力的高效模型。其包含两大创新:高分辨率感知(HRP)与高分辨率增强(HRE)。HRP模块通过图像裁剪处理高分辨率输入,整合局部与全局特征以实现多粒度质量优化;HRE模块利用高分辨率图像中的细粒度信息增强掩码特征,使其更精确地对齐文本特征,从而提升分割精度。大量消融实验证明了各模块的有效性,跨多个基准数据集的全面实验也验证了HRSeg的卓越性能。
原文摘要 · Abstract (English)
The reasoning segmentation task involves segmenting objects within an image by interpreting implicit user instructions, which may encompass subtleties such as contextual cues and open-world knowledge. Despite significant advancements made by existing approaches, they remain constrained by low perceptual resolution, as visual encoders are typically pre-trained at lower resolutions. Furthermore, simply interpolating the positional embeddings of visual encoders to enhance perceptual resolution yields only marginal performance improvements while incurring substantial computational costs. To address this, we propose HRSeg, an efficient model with high-resolution fine-grained perception. It features two key innovations: High-Resolution Perception (HRP) and High-Resolution Enhancement (HRE). The HRP module processes high-resolution images through cropping, integrating local and global features for multi-granularity quality. The HRE module enhances mask features by integrating fine-grained information from high-resolution images, refining their alignment with text features for precise segmentation. Extensive ablation studies validate the effectiveness of our modules, while comprehensive experiments on multiple benchmark datasets demonstrate HRSeg's superior performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。