arXiv:2501.14889cs.LG2025-01被引 2

提出EASE框架,让特征空间优化更高效、更通用。

Iterative Feature Space Optimization through Incremental Adaptive Evaluation

  • 分步解耦特征与样本,减少评估偏差
  • 增量更新评估器,效率提升显著
  • 适用于多种模型,泛化能力强

迭代特征空间优化需系统评估并调整特征空间以提升下游任务表现。现有方法存在三大缺陷:1)忽略样本差异导致评估偏差;2)针对特定模型定制特征空间引发过拟合和泛化差;3)每次迭代需从头训练评估器,严重降低效率。为此,我们提出通用自适应特征空间评估器(EASE),包含两个核心组件:特征-样本子空间生成器与情境注意力评估器。前者通过识别预测相关特征和难评样本,解耦特征分布以缓解偏差;后者采用加权共享多头注意力机制,将特征空间演化模式编码为嵌入向量,实现增量式评估。评估器持续保留历史知识并吸收新信息,因连续特征空间共享部分结构。在十四组真实数据集上的实验表明,该框架有效提升优化效率与泛化性能。代码与数据已公开。

原文摘要 · Abstract (English)

Iterative feature space optimization involves systematically evaluating and adjusting the feature space to improve downstream task performance. However, existing works suffer from three key limitations:1) overlooking differences among data samples leads to evaluation bias; 2) tailoring feature spaces to specific machine learning models results in overfitting and poor generalization; 3) requiring the evaluator to be retrained from scratch during each optimization iteration significantly reduces the overall efficiency of the optimization process. To bridge these gaps, we propose a gEneralized Adaptive feature Space Evaluator (EASE) to efficiently produce optimal and generalized feature spaces. This framework consists of two key components: Feature-Sample Subspace Generator and Contextual Attention Evaluator. The first component aims to decouple the information distribution within the feature space to mitigate evaluation bias. To achieve this, we first identify features most relevant to prediction tasks and samples most challenging for evaluation based on feedback from the subsequent evaluator. This decoupling strategy makes the evaluator consistently target the most challenging aspects of the feature space. The second component intends to incrementally capture evolving patterns of the feature space for efficient evaluation. We propose a weighted-sharing multi-head attention mechanism to encode key characteristics of the feature space into an embedding vector for evaluation. Moreover, the evaluator is updated incrementally, retaining prior evaluation knowledge while incorporating new insights, as consecutive feature spaces during the optimization process share partial information. Extensive experiments on fourteen real-world datasets demonstrate the effectiveness of the proposed framework. Our code and data are publicly available.

特征优化增量学习评估器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。