无需验证集，通过相变理论自动选择最优稀疏性，提升模型可解释性。
Validation-Free Sparse Learning: A Phase Transition Approach to Feature Selection
- 基于相变理论设计稀疏学习新方法，避免传统交叉验证。
- 在真实数据上实现预测精度与特征稀疏性的良好平衡。
- 适用于线性、浅层及深层神经网络，支持多种损失函数。
人工智能日益增长的存储与计算开销呼唤更节约且可解释的模型。稀疏模型（如线性模型、神经网络）通过仅保留相关特征，降低复杂度、防止过拟合并增强可解释性，是迈向真正智能AI的重要一步。稀疏程度的‘最佳值’具有主观性。本文提出一种新范式——此前仅在无噪声线性模型的压缩感知中被观察和数学研究过的特征恢复概率相变。我们展示了如何在实践中为一类稀疏学习器实现该相变。该方法灵活，适用于从线性到浅层及深层人工神经网络的复杂模型，并兼容多种损失函数和稀疏性促进惩罚项。其唯一正则化参数无需依赖交叉验证或验证集进行选择。在真实数据上，该方法提供了预测准确率与特征稀疏性之间的良好权衡。相关代码已开源：https://github.com/VcMaxouuu/HarderLASSO，包含所有模拟与可直接使用的模型。
原文摘要 · Abstract (English)
The growing environmental footprint of artificial intelligence (AI), especially in terms of storage and computation, calls for more frugal and interpretable models. Sparse models (e.g., linear, neural networks) offer a promising solution by selecting only the most relevant features, reducing complexity, preventing over-fitting and enabling interpretation-marking a step towards truly intelligent AI. The concept of a right amount of sparsity (without too many false positive or too few true positive) is subjective. So we propose a new paradigm previously only observed and mathematically studied for compressed sensing (noiseless linear models): obtaining a phase transition in the probability of retrieving the relevant features. We show in practice how to obtain this phase transition for a class of sparse learners. Our approach is flexible and applicable to complex models ranging from linear to shallow and deep artificial neural networks while supporting various loss functions and sparsity-promoting penalties. It does not rely on cross-validation or on a validation set to select its single regularization parameter. For real-world data, it provides a good balance between predictive accuracy and feature sparsity. A Python package is available at https://github.com/VcMaxouuu/HarderLASSO containing all the simulations and ready-to-use models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。