让AI像人一样分三步推理社会场景,提升理解与安全判断能力。
Cognitive Chain-of-Thought (CoCoT): Structured Multimodal Reasoning about Social Situations
- 分感知、情境、规范三阶段,结构化视觉语言推理。
- 多任务测试平均提升4.6%~5.9%,社交推理更准确。
- 训练中学习此结构后,推理无需提示也能提升5-6%。
链式思考(CoT)提示帮助模型逐步推理,但在需同时感知、理解与判断的社会视觉任务中易失效。为解决这一问题,本文提出认知链式思考(CoCoT),一种基于认知启发的三阶段框架:感知(提取事实)、情境(推断状态)、规范(应用社会规范)。在多模态意图消歧、心智理论、社会常识推理及安全指令遵循等任务上,模型性能平均提升4.6%至5.9%。进一步实验表明,对模型进行带有CoCoT结构的监督微调后,即使不使用提示,推理性能仍提升5%-6%,说明模型内化了该推理模式而非仅依赖指令。此结构显著提升可解释性与社会对齐性,为更可靠的多模态系统奠定基础。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) prompting helps models think step by step. But naive CoT breaks down in visually grounded social tasks, where models must perceive, understand, and judge all at once; bridging perception with norm-grounded reasoning. Recent work has introduced structured reasoning for multi-turn agent planning and visual QA, decomposing tasks into sequential sub-goals. To extend this to single-shot multimodal social reasoning, we introduce Cognitive Chain-of-Thought (CoCoT), a reasoning framework that structures vision-language-model (VLM) reasoning through three cognitively inspired stages: Perception (extract grounded facts), Situation (infer situations), and Norm (applying social norms). Evaluation across multiple distinct tasks such as multimodal intent disambiguation, multimodal theory of mind, social commonsense reasoning, and safety instruction following, shows consistent improvements (5.9% to 4.6% on average). We further explore the utility of CoCoT for improving models' reasoning through training and show that supervised fine-tuning on CoCoT-structured traces yields 5-6% improvements without explicit CoCoT prompting at inference, demonstrating that models internalize the structured reasoning pattern rather than merely following instructions. We show that structuring model reasoning through cognitively grounded stages enhances interpretability and social alignment, laying the groundwork for more reliable multimodal systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。