提出新框架，让模型既说清图像变化又精准定位位置。
CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation
- 双链架构分离语义理解与空间分割，提升推理精度。
- 在真实和合成数据集上均达当前最佳性能。
- 适合需要精细变化检测的视觉分析任务。
理解成对图像间的细微变化对于监控和图像编辑等任务至关重要。传统图像变化描述（ICC）方法缺乏空间定位能力，限制了其精度。为此，我们提出图像变化描述与分割（ICCS）这一新多模态任务，要求同时生成结构化的变化描述并实现像素级定位。为解决ICCS，我们设计了感知变化的描述与推理链（CCRC），采用双链架构，将语义推理与空间分割解耦。第一链Chain-of-Change-Captioning（CCC）通过在多模态大模型（MLLM）视觉与语言组件间插入多头变化感知注意力模块，增强细粒度变化感知，并判断变化是否可分割。若不可分割，则仅由CCC生成描述；否则激活第二链Chain-of-Change-Segmenting（CCS），利用CCC提供的空间先验，并通过变化感知标记精炼器优化掩码，实现边界精准定位。我们在带有像素级监督的合成与真实世界变化检测基准上评估了CCRC，实验表明其达到当前最优性能。
原文摘要 · Abstract (English)
Understanding and localizing subtle changes between paired images is critical for tasks such as surveillance and image editing. However, traditional Image Change Captioning (ICC) methods lack spatial grounding, limiting their precision. We introduce Image Change Captioning and Segmentation (ICCS), a new multimodal task that jointly requires structured change description and pixel-level localization. To address ICCS, we propose the Change-aware Captioning and Reasoning Chain (CCRC), a dual-chain framework that decouples semantic reasoning from spatial segmentation. The first chain, Chain-of-Change-Captioning (CCC), enhances fine-grained change perception via a visual fusion module based on Multi-Head Change-aware Attention inserted between the visual and language components of a Multimodal Large Language Model (MLLM). CCC also determines whether a change is segmentable. If not, it alone generates the caption. Otherwise, the second chain, Chain-of-Change-Segmenting (CCS), is activated, leveraging spatial priors from CCC and refining masks with a Change-aware Token Refiner for accurate boundary localization. We evaluate CCRC on both synthetic and real-world change detection benchmarks with pixel-level supervision. Experiments show CCRC achieves state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。