arXiv:2508.01984cs.CV2025-08ICCV被引 7

用隐式程序引导推理,让动作问答更灵活准确

IMoRe: Implicit Program-Guided Reasoning for Human Motion Q&A

  • 不依赖人工设计模块,通过结构化程序函数引导推理
  • 在Babel-QA和HuMMan新数据集上达到顶尖性能
  • 适合需要跨类型动作问答的科研与应用开发者

现有动作问答方法依赖显式程序执行,需手动定义功能模块,限制了可扩展性和适应性。为此,我们提出隐式程序引导的动作推理框架IMoRe,无需人工设计模块即可统一处理多种查询类型。不同于仅从问题词推断推理操作的隐式方法,本模型直接以结构化程序函数为条件,确保推理步骤更精确。此外,引入程序引导阅读机制,动态选择预训练动作视觉变换器(ViT)中的多层级动作表示,捕捉高层语义与细粒度运动线索。推理模块迭代优化记忆表示,利用结构化程序函数提取不同查询类型的关联信息。模型在Babel-QA上表现领先,并在基于HuMMan的新构建动作问答数据集上展现良好泛化能力。代码与数据集已开源。

原文摘要 · Abstract (English)

Existing human motion Q\&A methods rely on explicit program execution, where the requirement for manually defined functional modules may limit the scalability and adaptability. To overcome this, we propose an implicit program-guided motion reasoning (IMoRe) framework that unifies reasoning across multiple query types without manually designed modules. Unlike existing implicit reasoning approaches that infer reasoning operations from question words, our model directly conditions on structured program functions, ensuring a more precise execution of reasoning steps. Additionally, we introduce a program-guided reading mechanism, which dynamically selects multi-level motion representations from a pretrained motion Vision Transformer (ViT), capturing both high-level semantics and fine-grained motion cues. The reasoning module iteratively refines memory representations, leveraging structured program functions to extract relevant information for different query types. Our model achieves state-of-the-art performance on Babel-QA and generalizes to a newly constructed motion Q\&A dataset based on HuMMan, demonstrating its adaptability across different motion reasoning datasets. Code and dataset are available at: https://github.com/LUNAProject22/IMoRe.

动作问答隐式推理视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。