用大模型进化自动优化结构化数据预处理,提升预测准确率与可解释性。
Evolutionary Feature Engineering for Structured Data

- 将数据变换表示为标准接口的Python程序,通过大模型演化搜索最优预处理方案。
- 在时间序列上降低3%以上误差,新冠死亡数据集最高提升19%,适配最新时序模型。
- 适用于决策树等可解释模型,生成简洁有效特征,兼顾性能与透明度。
大语言模型正被用作演化优化中的开放搜索算子。本文提出演化特征工程(EFE),一种利用基于LLM的演化来发现结构化数据预处理变换的框架。EFE将变换表示为具有标准化fit/transform接口的Python程序,可直接嵌入现有机器学习流水线。演化过程中,候选程序通过数据上下文、统计摘要和验证集上的下游性能反馈进行优化。我们在两种场景下实现EFE:在时间序列预测中,EFE-Time学习可逆的、数据特定的归一化方法,显著提升现成时序基础模型表现;在多个数据集上平均降低3%或更多预测误差(MASE、WQL、MAE),在新冠死亡数据集上最高提升达19%。值得注意的是,该改进适用于如Chronos-2等近期时序模型。在表格预测中,EFE-Tab演化出紧凑的特征程序,增加有用可解释特征并剔除冗余项,性能优于或匹配现有基于LLM的特征工程方法。尤其对经典决策树效果显著,少量演化特征即可获得竞争力精度,同时保持可解释性。总体表明,基于LLM的演化能自动提升结构化数据处理的准确性与可解释性。
原文摘要 · Abstract (English)
Large language models are increasingly used as open-ended search operators in evolutionary optimization. We introduce Evolutionary Feature Engineering (EFE), a framework for using LLM-based evolution to discover preprocessing transformations for structured data. EFE represents transformations as Python programs with a standardized fit/transform interface, allowing them to be inserted directly into existing machine learning pipelines. During evolution, candidate programs are refined using dataset context, summary statistics, and downstream performance feedback on validation set. We instantiate EFE in two settings. For time-series forecasting, EFE-Time learns invertible, dataset-specific normalizations that improve off-the-shelf time-series foundation models. It reduces forecasting errors (MASE, WQL, MAE) 3% or more when averaged across datasets and improvements are as much as 19% on the COVID-Deaths dataset. Notably, these improvements occur with recent TSFMs such as Chronos-2. For tabular prediction, EFE-Tab evolves compact feature programs that add useful interpretable features and remove redundant ones, improving or matching existing LLM-based feature-engineering methods. We found EFE-Tab to be particularly effective on classical decision trees, where small sets of evolved features yield competitive accuracy while preserving interpretability. Overall, EFE demonstrates that LLM-based evolution can improve both accuracy and interpretability when automatically tackling structured data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。