用大模型零样本构建可解释决策树，无需标注数据
Zero-Shot Decision Tree Construction via Large Language Models
- 利用预训练大模型的先验知识，零样本完成属性离散化与基尼指数计算
- 在多个表格数据集上表现优于其他零样本方法，接近有监督树模型性能
- 适合数据稀缺场景，提供透明可解释的模型，适用于可信AI领域
本文提出一种基于大语言模型（LLMs）的零样本决策树构建算法，遵循分类与回归树（CART）原理。传统决策树依赖标注数据，通过信息增益或基尼指数等指标递归划分数据。本文方法则利用大模型中嵌入的预训练知识，在无训练数据条件下构建决策树。该方法借助大模型完成属性离散化、概率计算及基于概率的基尼指数计算等关键操作。实验表明，该零样本决策树在多个表格数据集上优于现有零样本基线方法，且性能可与监督学习驱动的决策树相媲美。所构建的决策树具备透明可解释性，在数据稀缺情况下仍能保持模型可读性，为低数据机器学习建立新基准，提供一种基于知识而非数据的树结构构建范式。
原文摘要 · Abstract (English)
This paper introduces a novel algorithm for constructing decision trees using large language models (LLMs) in a zero-shot manner based on Classification and Regression Trees (CART) principles. Traditional decision tree induction methods rely heavily on labeled data to recursively partition data using criteria such as information gain or the Gini index. In contrast, we propose a method that uses the pre-trained knowledge embedded in LLMs to build decision trees without requiring training data. Our approach leverages LLMs to perform operations essential for decision tree construction, including attribute discretization, probability calculation, and Gini index computation based on the probabilities. We show that these zero-shot decision trees can outperform baseline zero-shot methods and achieve competitive performance compared to supervised data-driven decision trees on tabular datasets. The decision trees constructed via this method provide transparent and interpretable models, addressing data scarcity while preserving interpretability. This work establishes a new baseline in low-data machine learning, offering a principled, knowledge-driven alternative to data-driven tree construction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。