用拓扑进化框架让大模型更智能地生成多样特征组合。
TopoFE: topology-aware LLM-guided Automated Feature Engineering
- 分家族探索+自适应记忆提示，避免重复搜索。
- 在29个数据集上性能超越现有方法，特征多样性提升显著。
- 适合需要高泛化特征工程的科研与工业场景。
表格学习中的自动特征工程可自然建模为程序合成问题，目标是从指数级搜索空间中发现预测性特征变换。近年来，大语言模型（LLM）通过突破预定义算子库限制，提升了AutoFE的表达能力。但现有基于LLM的方法仍受限于无状态生成和同质化搜索：特征建议仅基于静态提示，缺乏搜索经验积累；单群体探索易快速收敛至主流变换模式，难以发现跨变换家族的互补组合。本文提出拓扑感知的多岛进化框架TOPOFE，结合家族特化探索、自适应提示记忆与拓扑引导的知识迁移，高效发现多样且可组合的特征程序。在29个公开表格数据集上的实验表明，TOPOFE在分类与回归任务中持续优于当前最优AutoFE方法。除预测性能外，其生成的特征程序更具多样性与可迁移性，能在多个下游预测器及不同LLM底座间良好泛化。
原文摘要 · Abstract (English)
Automatic feature engineering (AutoFE) for tabular learning can be naturally formulated as a program synthesis problem, where the objective is to discover predictive feature transformations from an exponentially large search space. Recent advances in large language models (LLMs) have expanded the expressiveness of AutoFE by enabling feature program generation beyond predefined operator libraries. However, existing LLM-based approaches remain fundamentally limited by stateless generation and homogeneous search: feature proposals are produced from static prompts without accumulating search experience, while single-population exploration quickly converges to dominant transformation patterns and rarely discovers complementary feature compositions across transformation families. We propose TOPOFE, a topology-aware multi-island evolutionary framework for LLM-guided feature engineering. TOPOFE combines family-specialized exploration, adaptive prompt memory, and topology-guided knowledge transfer to efficiently discover diverse and compositional feature programs. Experiments on 29 public tabular datasets demonstrate consistent improvements over state-of-the-art AutoFE methods across classification and regression tasks. Beyond predictive performance, TOPOFE discovers more diverse and transferable feature programs that generalize across multiple downstream predictors and LLM backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。