提出可微分的特征选择方法，更好学习真实相关特征。
A Differentiable Rank-Based Objective For Better Feature Learning
- 用可微分近似替代非参数相关性度量，实现端到端训练。
- 在变量选择和卷积网络中有效减少虚假相关，提升特征质量。
- 适合需要可控特征学习的场景，如公平性分类任务。
本文借助现有统计方法深入理解数据驱动的特征学习。针对无模型变量选择方法 FOCI（基于非参数条件依赖系数），我们提出其参数化、可微分的近似版本。利用该近似相关系数，我们构建了新算法 difFOCI，因其可微特性与可学习参数，适用于更广泛的机器学习问题。我们在三类场景中验证：（1）作为变量选择方法，与 FOCI 基线对比；（2）作为神经网络参数化模型；（3）作为通用神经网络正则项，改善特征学习并抑制虚假相关。实验涵盖从简单玩具数据的变量选择到卷积网络显著性图的比较，并展示了 difFOCI 在公平分类中避免依赖敏感数据的应用潜力。
原文摘要 · Abstract (English)
In this paper, we leverage existing statistical methods to better understand feature learning from data. We tackle this by modifying the model-free variable selection method, Feature Ordering by Conditional Independence (FOCI), which is introduced in \cite{azadkia2021simple}. While FOCI is based on a non-parametric coefficient of conditional dependence, we introduce its parametric, differentiable approximation. With this approximate coefficient of correlation, we present a new algorithm called difFOCI, which is applicable to a wider range of machine learning problems thanks to its differentiable nature and learnable parameters. We present difFOCI in three contexts: (1) as a variable selection method with baseline comparisons to FOCI, (2) as a trainable model parametrized with a neural network, and (3) as a generic, widely applicable neural network regularizer, one that improves feature learning with better management of spurious correlations. We evaluate difFOCI on increasingly complex problems ranging from basic variable selection in toy examples to saliency map comparisons in convolutional networks. We then show how difFOCI can be incorporated in the context of fairness to facilitate classifications without relying on sensitive data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。