用大模型+传统方法做特征选择，效果更优。
LLM4FS: Leveraging Large Language Models for Feature Selection
- 结合大模型语义理解与传统统计方法选特征。
- 混合策略在多个数据集上优于纯大模型或传统方法。
- 适合需要高可靠特征选择的科研与工程场景。
大语言模型（LLMs）的进展为决策任务带来了新机遇，尤其在自动化特征选择方面。本文首先系统评估了当前最先进的DeepSeek-R1、GPT-o3-mini和GPT-4.5等基于LLM的特征选择方法。随后提出一种新型混合策略LLM4FS，将大模型与传统数据驱动方法相结合：输入数据样本至大模型，再调用随机森林、前向逐步选择等传统方法。分析表明，该混合策略融合了大模型的上下文理解能力与传统方法的高统计可靠性，在多个数据集上表现优异，甚至超越单独使用大模型或传统方法。最后指出其在决策应用中的局限性。代码已开源：https://github.com/xianchaoxiu/LLM4FS。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have provided new opportunities for decision-making, particularly in the task of automated feature selection. In this paper, we first comprehensively evaluate LLM-based feature selection methods, covering the state-of-the-art DeepSeek-R1, GPT-o3-mini, and GPT-4.5. Then, we propose a new hybrid strategy called LLM4FS that integrates LLMs with traditional data-driven methods. Specifically, input data samples into LLMs, and directly call traditional data-driven techniques such as random forest and forward sequential selection. Notably, our analysis reveals that the hybrid strategy leverages the contextual understanding of LLMs and the high statistical reliability of traditional data-driven methods to achieve excellent feature selection performance, even surpassing LLMs and traditional data-driven methods. Finally, we point out the limitations of its application in decision-making. Our code is available at https://github.com/xianchaoxiu/LLM4FS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。