用语言推理替代几何建模，让多模态大模型无需3D信息也能搞懂空间关系。
EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs
- 通过角色扮演式描述构建跨帧连贯语义场景图
- 仅用5000条SFT和2万条RL样本即达基准水平
- 适合想提升模型空间推理能力的研究者
多模态大语言模型在空间认知任务中应用日益广泛，需理解并交互复杂环境。现有方法多依赖3D先验或几何监督，虽提升性能但数据准备与对齐成本高；纯2D方法则因难以捕捉跨帧空间关系而表现受限。为此，我们提出EgoMind，一种链式思维框架，通过角色扮演式描述（Role-Play Caption）构建跨帧连贯的语义场景图，并结合渐进式空间分析（Progressive Spatial Analysis）逐步推理任务相关问题。仅需5000条自动生成的SFT样本和2万条强化学习样本，EgoMind在VSI-Bench、SPAR-Bench、SITE-Bench和SPBench上均取得有竞争力的表现，验证了语言推理在增强多模态大模型空间推理能力方面的有效性，展现了无几何依赖空间认知的潜力。代码与数据已开源。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) are increasingly being applied to spatial cognition tasks, where they are expected to understand and interact with complex environments. Most existing works improve spatial reasoning by introducing 3D priors or geometric supervision, which enhances performance but incurs substantial data preparation and alignment costs. In contrast, purely 2D approaches often struggle with multi-frame spatial reasoning due to their limited ability to capture cross-frame spatial relationships. To address these limitations, we propose EgoMind, a Chain-of-Thought framework that enables geometry-free spatial reasoning through Role-Play Caption, which jointly constructs a coherent linguistic scene graph across frames, and Progressive Spatial Analysis, which progressively reasons toward task-specific questions. With only 5K auto-generated SFT samples and 20K RL samples, EgoMind achieves competitive results on VSI-Bench, SPAR-Bench, SITE-Bench, and SPBench, demonstrating its effectiveness in strengthening the spatial reasoning capabilities of MLLMs and highlighting the potential of linguistic reasoning for spatial cognition. Code and data are released at https://github.com/Hyggge/EgoMind.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。