让AI看图推理更全面更连贯，通过多区域协作提升复杂视觉理解能力。
Watch Wider and Think Deeper: Collaborative Cross-modal Chain-of-Thought for Complex Visual Reasoning
- 动态定位图像关键区域，根据问题自适应选择信息点
- 跨区域协同推理，使思维链逻辑更连贯，准确率提升15.4%
- 适合需要深度视觉推理的科研与应用开发者
多模态推理需无缝融合视觉与语言线索，但现有思维链方法在跨模态场景中存在两大缺陷：（1）过度依赖单一粗粒度图像区域，（2）推理步骤间语义碎片化。为此，我们提出CoCoT（协同跨模态思维）框架，包含两项核心创新：a）动态多区域定位，根据问题自适应检测最相关图像区域；b）关系感知推理，通过迭代对齐视觉线索实现多区域协作，构建连贯逻辑链。基于此，我们构建了CoCoT-70K数据集，包含74,691个高质量样本，含多区域标注与结构化推理链。大量实验表明，CoCoT显著提升复杂视觉推理性能，在六个挑战性基准上，对LLaVA-1.5平均准确率提升15.4%，对Qwen2-VL提升4.0%。代码与数据已开源：https://github.com/deer-echo/CoCoT。
原文摘要 · Abstract (English)
Multi-modal reasoning requires the seamless integration of visual and linguistic cues, yet existing Chain-of-Thought methods suffer from two critical limitations in cross-modal scenarios: (1) over-reliance on single coarse-grained image regions, and (2) semantic fragmentation between successive reasoning steps. To address these issues, we propose the CoCoT (Collaborative Coross-modal Thought) framework, built upon two key innovations: a) Dynamic Multi-Region Grounding to adaptively detect the most relevant image regions based on the question, and b) Relation-Aware Reasoning to enable multi-region collaboration by iteratively aligning visual cues to form a coherent and logical chain of thought. Through this approach, we construct the CoCoT-70K dataset, comprising 74,691 high-quality samples with multi-region annotations and structured reasoning chains. Extensive experiments demonstrate that CoCoT significantly enhances complex visual reasoning, achieving an average accuracy improvement of 15.4% on LLaVA-1.5 and 4.0% on Qwen2-VL across six challenging benchmarks. The data and code are available at: https://github.com/deer-echo/CoCoT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。