用神经网络估算互信息,精准筛选复杂依赖关系下的关键特征
MINERVA: Mutual Information Neural Estimation for Supervised Feature Selection
- 用神经网络拟合特征与目标间的互信息,实现高阶依赖建模
- 在合成数据和真实欺诈数据上达到精确解,优于传统方法
- 适合处理特征间非线性、高阶交互的复杂场景
现有特征过滤方法依赖统计上的成对依赖度量来建模特征与目标的关系,但当目标依赖于高阶特征交互而非单个特征贡献时,该方法可能失效。我们提出基于神经网络估计互信息的监督特征选择方法 MINERVA,通过神经网络参数化互信息近似,并设计含稀疏正则项的损失函数进行特征选择。方法采用两阶段流程,分离表示学习与特征选择,提升泛化能力并更准确表达特征重要性。我们展示了文献中罕见的普遍依赖结构,并证明该方法通过集成评估特征子集,能有效捕捉这些复杂特征-目标关系。在合成数据和真实欺诈数据集上的实验结果表明,该方法具有优异性能并能获得精确解。
原文摘要 · Abstract (English)
Existing feature filters rely on statistical pair-wise dependence metrics to model feature-target relationships, but this approach may fail when the target depends on higher-order feature interactions rather than individual contributions. We introduce Mutual Information Neural Estimation Regularized Vetting Algorithm (MINERVA), a novel approach to supervised feature selection based on neural estimation of mutual information between features and targets. We paramaterize the approximation of mutual information with neural networks and perform feature selection using a carefully designed loss function augmented with sparsity-inducing regularizers. Our method is implemented in a two-stage process to decouple representation learning from feature selection, ensuring better generalization and a more accurate expression of feature importance. We present examples of ubiquitous dependency structures that are rarely captured in literature and show that our proposed method effectively captures these complex feature-target relationships by evaluating feature subsets as an ensemble. Experimental results on synthetic and real-life fraud datasets demonstrate the efficacy of our method and its ability to perform exact solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。