用大模型动态融合多个模型预测,提升表格数据预测精度。
Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboard
- 基于大模型的上下文集成框架,实现实例级自适应融合。
- 在多个表格数据集上超越主流基线与传统集成方法。
- 适合希望提升模型性能且无需专家知识的实践者。
表格数据是机器学习中的基础数据格式,广泛应用于竞赛和实际场景。由于特征分布和任务特性差异,梯度提升树和神经网络等模型在不同数据集上的表现差异显著,通常需依赖专家知识才能达到最优。现有集成方法多采用静态组合规则,缺乏对具体实例的适应性。本文提出一种基于大语言模型(LLM)的上下文集成框架,通过测试实例的近邻样本及其外部模型的预测结果构建上下文,并引入链式表格思维(CoT²)提示策略,引导大模型进行多步可解释推理,实现动态、实例级的预测融合。实验表明,该方法在多个表格数据集上优于经过调优的基线及标准集成技术。
原文摘要 · Abstract (English)
Tabular data, a fundamental data format in machine learning, is predominantly utilized in competitions and real-world applications. The performance of tabular models--such as gradient boosted decision trees and neural networks--can vary significantly across datasets due to differences in feature distributions and task characteristics. Achieving top performance on each dataset often requires specialized expert knowledge. To address this variability, practitioners often aggregate the predictions of multiple models. However, conventional aggregation strategies typically rely on static combination rules and lack instance-level adaptability. In this work, we propose an in-context ensemble framework for tabular prediction that leverages large language models (LLMs) to perform dynamic, instance-specific integration of external model predictions. Without access to raw tabular features or semantic information, our method constructs a context around each test instance using its nearest neighbors and the predictions from a pool of external models. Within this enriched context, we introduce Chain of Tabular Thoughts (CoT$^2$), a prompting strategy that guides LLMs through multi-step, interpretable reasoning, making still further progress toward expert-level decision-making. Experimental results show that our method outperforms well-tuned baselines and standard ensemble techniques across a wide range of tabular datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。