首个可证明控制假阳性率的深度特征选择方法,适用于多种网络结构。
Provable FDR Control for Deep Feature Selection: Deep MLPs and Beyond
- 基于梯度重要性向量与渐近正态性,实现对特征选择中假阳性率的理论控制。
- 在特征数远大于潜变量数的渐近条件下,验证了方法对假阳性率的有效控制。
- 适合关注模型可解释性与统计保证的深度学习研究者使用。
我们提出一种灵活的基于深度神经网络的特征选择框架,可近似控制假发现率(FDR),即第一类错误率。该方法适用于第一层为全连接的网络架构,从第二层起可兼容任意宽度和深度的多层感知机(MLP)、卷积与循环网络、注意力机制、残差连接及丢弃(dropout)。同时支持独立于数据的初始化与学习率的随机梯度下降。据我们所知,这是首个在如此广泛深度学习设置下提供特征选择中FDR控制理论保证的工作。分析基于多指标数据生成模型,在特征维度 $n$ 比潜变量维度 $q^{*}$ 发散更快的渐近条件下进行,样本量、训练迭代次数、网络深度及隐藏层宽度均不限制。在此设定下,我们证明梯度特征重要性向量的每个坐标均服从边际正态近似,从而支持渐近FDR控制的有效性。理论限制在于假设设计矩阵满足 $\ extbf{B}$-右正交不变性,文中也讨论了更广的推广可能。数值实验进一步验证了理论结论。
原文摘要 · Abstract (English)
We develop a flexible feature selection framework based on deep neural networks that approximately controls the false discovery rate (FDR), a measure of Type-I error. The method applies to architectures whose first layer is fully connected. From the second layer onward, it accommodates multilayer perceptrons (MLPs) of arbitrary width and depth, convolutional and recurrent networks, attention mechanisms, residual connections, and dropout. The procedure also accommodates stochastic gradient descent with data-independent initializations and learning rates. To the best of our knowledge, this is the first work to provide a theoretical guarantee of FDR control for feature selection within such a general deep learning setting. Our analysis is built upon a multi-index data-generating model and an asymptotic regime in which the feature dimension $n$ diverges faster than the latent dimension $q^{*}$, while the sample size, the number of training iterations, the network depth, and hidden layer widths are left unrestricted. Under this setting, we show that each coordinate of the gradient-based feature-importance vector admits a marginal normal approximation, thereby supporting the validity of asymptotic FDR control. As a theoretical limitation, we assume $\mathbf{B}$-right orthogonal invariance of the design matrix, and we discuss broader generalizations. We also present numerical experiments that underscore the theoretical findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。