首个包含完整归约证明的复杂推理数据集，用于评测大模型数学推理能力。
The Karp Dataset
- 构建首个包含详细NP完全性归约证明的数据集，覆盖从本科习题到论文难题的难度梯度。
- 评估顶尖模型在该任务上的表现，发现微调后推理能力显著提升。
- 适合研究大模型数学推理、形式化证明与算法分析的学者和工程师。
理解大型语言模型（LLMs）的数学推理能力是人工智能研究的核心议题。这一新领域需要为训练和基准测试构建推理任务数据集。为此，我们引入了Karp数据集：首个包含详细NP完全性归约证明的数据集。这些归约难度各异，涵盖本科课程中的简单练习到学术论文中的复杂归约。我们评估了当前最优模型在此任务上的表现，并展示了使用Karp数据集进行微调对模型推理能力的提升效果。
原文摘要 · Abstract (English)
Understanding the mathematical reasoning capabilities of Large Language Models (LLMs) is a central topic in the study of artificial intelligence. This new domain necessitates the creation of datasets of reasoning tasks for both training and benchmarking the performance of LLMs. To this end, we introduce the Karp dataset: The first dataset composed of detailed proofs of NP-completeness reductions. The reductions vary in difficulty, ranging from simple exercises of undergraduate courses to more challenging reductions from academic papers. We compare the performance of state-of-the-art models on this task and demonstrate the effect of fine-tuning with the Karp dataset on reasoning capacity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。