arXiv:2505.15152cs.LG2025-05NeurIPS被引 11

用扩散模型生成任务最优特征,提升预测准确率与效率

Sculpting Features from Noise: Reward-Guided Hierarchical Diffusion for Task-Optimal Feature Transformation

  • 将特征变换视为奖励引导的生成任务,结合变分自编码器与扩散模型
  • 在14个数据集上超越现有方法,训练和推理时间显著降低
  • 适合需要高效特征工程的机器学习应用

特征变换(FT)通过数学操作从原始特征构造新特征,以增强下游模型的表达能力。然而,现有方法存在明显局限:离散搜索受限于庞大的组合空间,难以实用;连续搜索对初始化和步长敏感,易陷入局部最优,限制全局探索。为此,DIFFT 将 FT 重新定义为奖励引导的生成任务。首先利用变分自编码器(VAE)学习特征集的紧凑且富有表现力的隐空间;随后,潜在扩散模型(LDM)在该空间中生成高质量特征嵌入,其轨迹由性能评估器引导,朝向特定任务的最优解。这一结合全局分布学习(来自 LDM)与目标优化(奖励引导)的策略,生成强健的嵌入表示,并通过新颖的半自回归解码器高效转化为结构化、离散特征,保留特征内依赖关系的同时支持特征间的并行生成。在14个基准数据集上的大量实验表明,DIFFT 在预测准确率和鲁棒性方面持续优于当前最优基线,且训练与推理时间显著更低。

原文摘要 · Abstract (English)

Feature Transformation (FT) crafts new features from original ones via mathematical operations to enhance dataset expressiveness for downstream models. However, existing FT methods exhibit critical limitations: discrete search struggles with enormous combinatorial spaces, impeding practical use; and continuous search, being highly sensitive to initialization and step sizes, often becomes trapped in local optima, restricting global exploration. To overcome these limitations, DIFFT redefines FT as a reward-guided generative task. It first learns a compact and expressive latent space for feature sets using a Variational Auto-Encoder (VAE). A Latent Diffusion Model (LDM) then navigates this space to generate high-quality feature embeddings, its trajectory guided by a performance evaluator towards task-specific optima. This synthesis of global distribution learning (from LDM) and targeted optimization (reward guidance) produces potent embeddings, which a novel semi-autoregressive decoder efficiently converts into structured, discrete features, preserving intra-feature dependencies while allowing parallel inter-feature generation. Extensive experiments on 14 benchmark datasets show DIFFT consistently outperforms state-of-the-art baselines in predictive accuracy and robustness, with significantly lower training and inference times.

特征工程扩散模型自动特征学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。