用超网络动态生成特征选择的分类器参数,提升模型泛化能力。
Hypernetworks for Dynamic Feature Selection

- 基于超网络按需生成特定特征子集的分类器参数。
- 在合成与真实表格数据上超越现有最佳方法,图像数据表现优异。
- 零样本泛化能力显著优于传统方法,适合资源受限场景。
动态特征选择(DFS)是在预算约束下为每个样本逐次获取特征的机器学习框架。特征获取路径呈指数级增长,导致模型难以兼顾特定场景拟合与通用性能,即使特征空间中等规模也是如此。本文分析现有DFS方法的结构局限性,并提出 extsc{Hyper-DFS},一种基于超网络的DFS方法,可按需生成针对特定特征子集的分类器参数。相比掩码嵌入方法,超网络具有更小的结构复杂度上界。同时采用Set Transformer编码构建平滑的条件空间,使功能相似的任务在几何上也相近。实验表明, extsc{Hyper-DFS}在合成与真实表格数据上全面优于现有先进方法,在所有测试图像数据集上表现竞争力或更优,并展现出远超现有方法的零样本泛化能力,能有效处理训练中未见过的特征子集。
原文摘要 · Abstract (English)
Dynamic feature selection (DFS) is a machine learning framework in which features are acquired sequentially for individual samples under budget constraints. The exponential growth in the number of possible feature acquisition paths forces a DFS model to balance fitting specific scenarios against maintaining general performance, even when the feature space is moderate in size. In this paper, we study the structural limitations of existing DFS approaches to achieve an optimal solution. Then, we propose \textsc{Hyper-DFS}, a hypernetwork-based DFS approach that generates feature subset-specific classifier parameters on demand. We show that the use of hypernetworks compared to mask-embedding methods results in a smaller structural complexity bound. We also use a Set Transformer encoding to create a smooth conditioning space for the hypernetwork, so that functionally similar tasks are also geometrically close. In our benchmarks, \textsc{Hyper-DFS} outperforms all state-of-the-art approaches on synthetic and real-life tabular data. It is also competitive or superior across all image datasets tested, and shows substantially stronger zero-shot generalisation to feature subsets never seen during training than existing DFS approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。