用可执行代码替代文本推理,让小模型更准地做金融数据计算。
Gold-Guided Programmatic Distillation for Financial Reasoning over Hybrid Tables and Text
- 用可验证的Python程序代替自然语言推理作为监督信号。
- 7B小模型在TAT-QA上达到87.00 EM,超越72B大模型。
- 适合需要高精度数值推理的金融问答场景。
针对混合表格与文本的金融问答任务,传统大模型生成的自然语言推理易出现算术错误,难以作为可靠的监督信号。本文提出一种基于程序化蒸馏的方法,利用执行验证的Python程序替代自由文本推理,从大教师模型中迁移可靠的数值推理能力。该方法通过黄金推导引导程序生成,并仅保留可正确执行且输出正确答案的程序,确保高质量监督。进一步引入迭代恢复阶段,重新处理教师失败的样本,使学生模型能学习到新验证的程序。在TAT-QA数据集上的实验表明,最优7B学生模型在测试集上取得87.00 EM / 87.18 F1,显著优于72B教师模型(78.46 EM)及主流基线如TAGOP和TAT-LLM。结果证明,执行验证的程序化蒸馏是训练小型模型实现可靠数值推理的有效且可扩展框架。
原文摘要 · Abstract (English)
Financial question answering over hybrid tabular and textual data may require multi-source reasoning and precise numerical computation. While large language models (LLMs) can generate intermediate reasoning steps, natural-language rationales remain prone to arithmetic errors, making them an unreliable supervision source for distillation. Building on programmatic distillation, we develop an approach that transfers reliable numerical reasoning from a large teacher model to a compact student using execution-verified Python programs instead of free-form textual rationales. It leverages gold derivations to guide teacher-side program synthesis and retains only programs that execute correctly and produce the gold answer, ensuring high-quality supervision. We further introduce an iterative recovery stage that revisits teacher-failed examples, enabling the student to recover and incorporate newly verified programs into training. Experiments on TAT-QA show that our framework is highly effective for hybrid financial reasoning. Our best 7B student achieves 87.00 EM / 87.18 F1 on the test set, substantially outperforming the 72B teacher (78.46 EM) as well as traditional and strong LLM-based baselines, including TAGOP and TAT-LLM. These results demonstrate that execution-verified programmatic distillation provides an effective and extensible framework for training smaller models to perform reliable numerical reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。