提出分步推理框架，让AI更懂复杂操作流程的下一步该做什么。
Chain-of-Procedure: Hierarchical Visual-Language Reasoning for Procedural QA
- 分层推理：先根据图片找相关步骤，再细化语义生成下一步
- 在6个模型上提升最高13%，显著改善流程理解能力
- 适合需要理解操作步骤的智能助手、自动化系统研发者
视觉语言模型在标准图像文本任务上表现优异，但在视觉程序问答（VP-QA）任务上的潜力尚未充分挖掘。该任务要求用户上传复杂流程中间状态的图片，以询问下一步操作。为此，我们构建了ProcedureVQA，一个专门针对视觉程序推理的新多模态基准。分析发现现有模型存在两大问题：基于视觉状态检索结构化流程的能力不足，以及图像序列粒度与文本步骤分解不匹配。为此，我们提出链式流程（Chain-of-Procedure, CoP）框架，通过视觉线索检索相关指令，再经语义分解进行步骤细化，最终生成下一动作。在六个视觉语言模型上的实验表明，CoP相较标准基线最多提升13%绝对性能。
原文摘要 · Abstract (English)
Recent advances in vision-language models (VLMs) have achieved impressive results on standard image-text tasks, yet their potential for visual procedure question answering (VP-QA) remains largely unexplored. VP-QA presents unique challenges where users query next-step actions by uploading images for intermediate states of complex procedures. To systematically evaluate VLMs on this practical task, we propose ProcedureVQA, a novel multimodal benchmark specifically designed for visual procedural reasoning. Through comprehensive analysis, we identify two critical limitations in current VLMs: inadequate cross-modal retrieval of structured procedures given visual states, and misalignment between image sequence granularity and textual step decomposition. To address these issues, we present Chain-of-Procedure (CoP), a hierarchical reasoning framework that first retrieves relevant instructions using visual cues, then performs step refinement through semantic decomposition, and finally generates the next step. Experiments across six VLMs demonstrate CoP's effectiveness, achieving up to 13% absolute improvement over standard baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。