arXiv:2509.11335cs.LGcond-mat.mtrl-sci2025-09被引 2

首个面向材料表征的多模态评测数据集,助力AI理解材料分析数据。

MatQnA: A Benchmark Dataset for Multi-modal Large Language Models in Materials Characterization and Analysis

  • 融合大模型与人工验证构建高质量问答对
  • 主流模型在客观题上准确率接近90%
  • 适合材料科学与AI交叉研究者使用

近年来,大语言模型(LLMs)在编程、写作等通用领域取得显著突破,并展现出在多种科研场景中的潜力。然而,人工智能模型在高度专业化的材料表征与分析领域的能力尚未得到系统性验证。为弥补这一空白,我们提出MatQnA,这是首个专为材料表征技术设计的多模态基准数据集。该数据集涵盖十种主流表征方法,包括X射线光电子能谱(XPS)、X射线衍射(XRD)、扫描电子显微镜(SEM)、透射电子显微镜(TEM)等。我们采用大模型结合人工验证的混合方法构建高质量的问答对,包含多项选择与主观问答。初步评估结果显示,当前最先进的多模态AI模型(如GPT-4.1、Claude 4、Gemini 2.5和Doubao Vision Pro 32K)在材料数据分析与解释任务的客观题上已达到近90%的准确率,显示出在材料表征与分析中应用的巨大潜力。MatQnA数据集已公开发布于https://huggingface.co/datasets/richardhzgg/matQnA。

原文摘要 · Abstract (English)

Recently, large language models (LLMs) have achieved remarkable breakthroughs in general domains such as programming and writing, and have demonstrated strong potential in various scientific research scenarios. However, the capabilities of AI models in the highly specialized field of materials characterization and analysis have not yet been systematically or sufficiently validated. To address this gap, we present MatQnA, the first multi-modal benchmark dataset specifically designed for material characterization techniques. MatQnA includes ten mainstream characterization methods, such as X-ray Photoelectron Spectroscopy (XPS), X-ray Diffraction (XRD), Scanning Electron Microscopy (SEM), Transmission Electron Microscopy (TEM), etc. We employ a hybrid approach combining LLMs with human-in-the-loop validation to construct high-quality question-answer pairs, integrating both multiple-choice and subjective questions. Our preliminary evaluation results show that the most advanced multi-modal AI models (e.g., GPT-4.1, Claude 4, Gemini 2.5, and Doubao Vision Pro 32K) have already achieved nearly 90% accuracy on objective questions in materials data interpretation and analysis tasks, demonstrating strong potential for applications in materials characterization and analysis. The MatQnA dataset is publicly available at https://huggingface.co/datasets/richardhzgg/matQnA.

材料表征多模态评测数据集大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。