用视觉语言模型评估分子级空间智能，发现当前模型远不如人类。
From Macro to Micro: Benchmarking Microscopic Spatial Intelligence on Molecules via Vision-Language Models
- 构建涵盖4000个分子的16万+问答对，测试模型对微观空间关系的理解能力
- 大模型微调后在空间变换任务上超越人类，但在氢键识别等科学任务表现差
- 强调融合领域知识是实现科学通用AI的关键，适合研究多模态与科学推理的学者
本文提出微观空间智能（MiSI）概念，即对不可见微观实体空间关系的感知与推理能力，这是科学发现的基础。为评估视觉语言模型（VLMs）在此领域的潜力，我们构建了系统性基准框架MiSI-Bench，包含约4000个分子结构生成的58.7万张图像和超过16.3万个问答对，覆盖九类互补任务，从基础空间变换到复杂关系识别。实验表明，当前最先进的VLMs在该基准上的表现显著低于人类水平。然而，经过微调的70亿参数模型展现出巨大潜力，在空间变换任务中甚至超过人类；其在氢键识别等科学任务中的薄弱表现则凸显了融入显式领域知识对迈向科学通用人工智能的必要性。数据集已公开于https://huggingface.co/datasets/zongzhao/MiSI-bench。
原文摘要 · Abstract (English)
This paper introduces the concept of Microscopic Spatial Intelligence (MiSI), the capability to perceive and reason about the spatial relationships of invisible microscopic entities, which is fundamental to scientific discovery. To assess the potential of Vision-Language Models (VLMs) in this domain, we propose a systematic benchmark framework MiSI-Bench. This framework features over 163,000 question-answer pairs and 587,000 images derived from approximately 4,000 molecular structures, covering nine complementary tasks that evaluate abilities ranging from elementary spatial transformations to complex relational identifications. Experimental results reveal that current state-of-the-art VLMs perform significantly below human level on this benchmark. However, a fine-tuned 7B model demonstrates substantial potential, even surpassing humans in spatial transformation tasks, while its poor performance in scientifically-grounded tasks like hydrogen bond recognition underscores the necessity of integrating explicit domain knowledge for progress toward scientific AGI. The datasets are available at https://huggingface.co/datasets/zongzhao/MiSI-bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。