arXiv:2505.11601cs.LGcs.AI2025-05KDD被引 7

提出新方法解决特征选择中嵌入不不变与搜索失效问题。

Continuous Optimization for Feature Selection with Permutation-Invariant Embedding and Policy-Guided Search

  • 用编码器-解码器结构和诱导点机制实现特征子集的排列不变嵌入
  • 通过策略强化学习在非凸空间中高效搜索,避免陷入局部最优
  • 在多个数据集上表现更优,适合复杂特征交互场景

特征选择通过剔除冗余特征提升下游任务性能与计算效率。现有方法难以捕捉复杂特征交互且适应性差。近期工作引入生成智能以揭示特征间复杂关系,但仍存在两大局限:1)连续空间中特征子集嵌入受排列敏感性影响,顺序变化会引入偏差,削弱嵌入学习;2)基于梯度的搜索依赖凸性假设,而该假设通常不成立,导致搜索效果下降、子集质量不佳。为此,本文提出新框架:1)在连续嵌入空间中保持特征子集知识并确保排列不变性;2)在不依赖强凸假设的前提下有效探索嵌入空间。针对第一目标,设计编码器-解码器范式,通过子集内成对关系捕捉特征交互,消除顺序影响,并引入诱导点机制加速成对关系计算。针对第二目标,采用基于策略的强化学习(RL)引导嵌入空间探索,代理能自适应优先高潜力区域,平衡多目标,避免局部最优。大量实验表明,所提模型在有效性、效率、鲁棒性和可解释性方面均优于现有方法。

原文摘要 · Abstract (English)

Feature selection removes redundant features to enhanc performance and computational efficiency in downstream tasks. Existing works often struggle to capture complex feature interactions and adapt to diverse scenarios. Recent advances in this domain have incorporated generative intelligence to address these drawbacks by uncovering intricate relationships between features. However, two key limitations remain: 1) embedding feature subsets in a continuous space is challenging due to permutation sensitivity, as changes in feature order can introduce biases and weaken the embedding learning process; 2) gradient-based search in the embedding space assumes convexity, which is rarely guaranteed, leading to reduced search effectiveness and suboptimal subsets. To address these limitations, we propose a new framework that can: 1) preserve feature subset knowledge in a continuous embedding space while ensuring permutation invariance; 2) effectively explore the embedding space without relying on strong convex assumptions. For the first objective, we develop an encoder-decoder paradigm to preserve feature selection knowledge into a continuous embedding space. This paradigm captures feature interactions through pairwise relationships within the subset, removing the influence of feature order on the embedding. Moreover, an inducing point mechanism is introduced to accelerate pairwise relationship computations. For the second objective, we employ a policy-based reinforcement learning (RL) approach to guide the exploration of the embedding space. The RL agent effectively navigates the space by balancing multiple objectives. By prioritizing high-potential regions adaptively and eliminating the reliance on convexity assumptions, the RL agent effectively reduces the risk of converging to local optima. Extensive experiments demonstrate the effectiveness, efficiency, robustness and explicitness of our model.

特征选择强化学习嵌入学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。