测试大模型理解古汉语数学题的能力，发现仍不理想。
Can reasoning models comprehend mathematical problems in Chinese ancient texts? An empirical study based on data from Suanjing Shishu
- 构建古籍数学基准数据集Guji_MATH，含538道题
- 六模型在古文题上表现逊于现代数学任务
- 适合研究古汉语理解与文化知识融合的学者
本研究针对中文古籍数学经典智能化处理难题，基于《算经十书》构建了Guji_MATH基准数据集。通过机器辅助标注与人工验证，从8部经典文献中提取538道数学题，形成以“问题-答案-解法”为核心的结构化数据集，并标注题型与难度等级。设计闭卷（自主求解）与开卷（复现古法）双评估模式，对六类主流推理模型进行测评。结果表明，模型能部分理解并求解古汉语数学题，但整体性能显著低于现代数学基准。提升模型对古汉语及传统文化知识的理解能力，是优化关键。研究为挖掘古籍数学知识、传播传统文化提供方法支持，并拓展了跨语言、跨文化推理能力评估的新视角。
原文摘要 · Abstract (English)
This study addresses the challenges in intelligent processing of Chinese ancient mathematical classics by constructing Guji_MATH, a benchmark for evaluating classical texts based on Suanjing Shishu. It systematically assesses the mathematical problem-solving capabilities of mainstream reasoning models under the unique linguistic constraints of classical Chinese. Through machine-assisted annotation and manual verification, 538 mathematical problems were extracted from 8 canonical texts, forming a structured dataset centered on the "Question-Answer-Solution" framework, supplemented by problem types and difficulty levels. Dual evaluation modes--closed-book (autonomous problem-solving) and open-book (reproducing classical solution methods)--were designed to evaluate the performance of six reasoning models on ancient Chinese mathematical problems. Results indicate that reasoning models can partially comprehend and solve these problems, yet their overall performance remains inferior to benchmarks on modern mathematical tasks. Enhancing models' classical Chinese comprehension and cultural knowledge should be prioritized for optimization. This study provides methodological support for mining mathematical knowledge from ancient texts and disseminating traditional culture, while offering new perspectives for evaluating cross-linguistic and cross-cultural capabilities of reasoning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。