用大模型生成新特征，提升分类效果
Feature Generation Using LLMs: An Evolutionary Algorithm Approach
- 基于提示词和数学逻辑运算，让大模型生成新特征
- 在8个不同数据集上，新特征显著提升分类性能
- 适合需要自动特征工程的机器学习任务
机器学习流程中，为每个实体生成能代表其特性的特征是关键步骤。特征工程有助于发现属性间的潜在关系，使原本难以处理的数据可被算法利用。大型语言模型在编程、数学推理和世界知识处理方面展现出潜力。本文提出一种基于大模型的表格数据特征生成方法：输入一组已有特征和提示词，由大模型生成新特征；再通过选择算法筛选出表现最佳的特征组合。我们在8个来自不同领域和数据类型的数据集上验证该方法，结果表明，大模型能生成基于数学与逻辑运算的新特征，且在多数情况下有效提升分类准确率。
原文摘要 · Abstract (English)
A crucial step in machine learning pipelines is to present each entity with features or attributes that are representative of the characteristics of the processed entities. Feature engineering is an important step in finding a relation among attributes that otherwise may not be processed by the ML algorithms. Meanwhile, Large Language Models have shown promising abilities in coding, mathematical reasoning, and processing world knowledge. In this work, we utilize an LLM for the problem of feature generation from tabular data based on the previously given features. We have created a pipeline that takes a set of attributes and a prompt to generate new features. Then, our selection algorithm selects the best-performing sets of attributes. We apply our method to eight datasets from different domains and data types. Our results show that, in most cases, the language model can produce new features based on mathematical and logical operators that are useful for the given tasks and can improve classification results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。