arXiv:2606.08295cs.CL2026-06

让大模型读懂表格数据并生成可解释的推理,效果接近顶尖树模型。

TLRD: Teaching LLMs to Reason over Tabular Data with Tri-Level Rationale Distillation

论文配图:TLRD: Teaching LLMs to Reason over Tabular Data with Tri-Level Rationale Distillation
图 1 · 摘自论文原文
  • 用三层次证据合成解释数据,指导大模型推理
  • 在多个领域数据集上逼近树集成模型性能
  • 适合需要可解释决策的高风险场景

表格数据是存储真实世界信息的主要形式,支撑众多机器学习工业应用。传统预测模型虽性能强,但无法提供可读的、针对具体案例的解释,影响决策可信度。大语言模型(LLMs)能自然生成预测与解释,但表格数据的特征分布和交互模式使模型理解困难,仅用标签微调会引发灾难性遗忘。为此,我们提出三层次推理蒸馏(TLRD)框架,将仅含标签的表格数据转化为结构化推理监督信号。TLRD利用高性能教师模型,在实例级特征、数据集级分布上下文和对比级检索邻居三个层面构建证据体系,再将推理过程蒸馏至学生大模型,实现仅凭原始特征即可零开销预测与可解释推理。在多个领域数据集上的实验表明,TLRD显著缩小了大模型与先进树集成模型之间的性能差距,同时生成有依据且易读的解释,为高风险决策提供可靠参考。

原文摘要 · Abstract (English)

Tabular data is a primary medium for storing real-world information, driving many industrial applications of machine learning. Traditional predictors achieve strong predictive performance but do not provide readable, case-specific explanations essential for decision-making. Large Language Models (LLMs) can naturally bridge this gap by generating predictions alongside explanations. However, dataset-specific patterns, such as feature distributions and interactions, make tabular data difficult for LLMs to understand and reason over, while label-only fine-tuning improves performance at the cost of catastrophic forgetting. To address this problem, we propose Tri-Level Rationale Distillation (TLRD), a framework that converts label-only tabular datasets into structured rationale supervision for LLMs. TLRD uses a high-capacity teacher to synthesize a rationale corpus grounded in three complementary levels of evidence: instance-level feature, dataset-level distributional context, and comparison-level retrieved neighbors, then distills the rationale into student LLMs, enabling zero-overhead prediction and grounded explanation from raw features only. Experiments on multiple domain datasets show that TLRD significantly closes the performance gap between LLMs and state-of-the-art tree ensembles while producing grounded and readable explanations, offering a valuable reference for high-stakes decision-making.

大模型表格推理可解释性知识蒸馏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。