用模板+大模型改写,生成高质量且多样的表格数学题。
Template-Driven LLM-Paraphrased Framework for Tabular Math Word Problem Generation
- 基于真实题目提取模板,确保生成题目的正确性。
- 通过大模型改写扩展,提升题目背景与表述多样性。
- 添加解题步骤说明,帮助模型更好理解推理过程。
解决表格型数学应用题(TMWP)已成为评估大语言模型(LLM)数学推理能力的关键任务,但高质量的TMWP数据集收集成本高、耗时长。现有自动生成方法常面临正确性或多样性不足的问题。为此,本文提出一种模板驱动的大模型改写框架(TeLL),可生成具有多样化背景、准确表格、问题、答案和解题过程的高质量样本。首先从真实样本中提取模板生成初始题目,保证正确性;再利用大模型对模板进行扩展与改写,实现多样性;同时发现推理标注对解题至关重要,因此在每道题解法中增加清晰的推理步骤。基于此框架,我们构建了新数据集TabMWP-TeLL,遵循原始TabMWP的数据类型分布,并在多种LLM上进行了广泛实验,验证了其在提升TMWP求解性能上的有效性。代码与数据已开源:https://github.com/Jason8Kang/TELL。
原文摘要 · Abstract (English)
Solving tabular math word problems (TMWPs) has become a critical role in evaluating the mathematical reasoning ability of large language models (LLMs), where large-scale TMWP samples are commonly required for LLM fine-tuning. Since the collection of high-quality TMWP datasets is costly and time-consuming, recent research has concentrated on automatic TMWP generation. However, current generated samples usually suffer from issues of either correctness or diversity. In this paper, we propose a Template-driven LLM-paraphrased (TeLL) framework for generating high-quality TMWP samples with diverse backgrounds and accurate tables, questions, answers, and solutions. To this end, we first extract templates from existing real samples to generate initial problems, ensuring correctness. Then, we adopt an LLM to extend templates and paraphrase problems, obtaining diverse TMWP samples. Furthermore, we find the reasoning annotation is important for solving TMWPs. Therefore, we propose to enrich each solution with illustrative reasoning steps. Through the proposed framework, we construct a high-quality dataset TabMWP-TeLL by adhering to the question types in the TabMWP dataset, and we conduct extensive experiments on a variety of LLMs to demonstrate the effectiveness of TabMWP-TeLL in improving TMWP solving performance. The code and data of this paper are available at: https://github.com/Jason8Kang/TELL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。