用扩散模型选稳定特征,让模型在数据变化时依然靠谱。
Causally-Guided Diffusion for Stable Feature Selection
- 用扩散模型生成特征选择的先验,结合跨环境稳定性目标。
- 在多个真实数据集上,选出的特征更稳定、泛化能力更强。
- 适合追求鲁棒性的机器学习研究者和工业应用落地场景。
特征选择是稳健数据驱动AI的基础,但现有方法多在单一数据分布下优化预测性能,常选中虚假特征,在分布漂移时失效。受因果不变性启发,本文从稳定性视角出发,提出因果引导扩散特征选择(CGDFS)。将特征选择建模为特征子集的近似后验推断,后验质量同时偏好低预测误差和跨环境低方差。框架融合三大核心:1)将特征选择形式化为感知稳定性的后验采样,以因果不变性作为软归纳偏置;2)训练扩散模型作为连续选择掩码的先验,结合关注环境间一致性的似然函数,捕捉特征间结构依赖,实现对组合爆炸式选择空间的可扩展探索;3)采用引导退火朗之万采样,联合扩散先验与稳定性目标,实现可计算、带不确定性的后验推断,避免离散优化,得到鲁棒特征选择。在存在分布偏移的真实开源数据集上评估,无论分类还是回归任务,CGDFS均持续选出更稳定、可迁移的特征子集,显著提升分布外性能并优于稀疏性、树模型及稳定性选择等基线方法。
原文摘要 · Abstract (English)
Feature selection is fundamental to robust data-centric AI, but most existing methods optimize predictive performance under a single data distribution. This often selects spurious features that fail under distribution shifts. Motivated by principles from causal invariance, we study feature selection from a stability perspective and introduce Causally-Guided Diffusion for Stable Feature Selection (CGDFS). In CGDFS, we formalized feature selection as approximate posterior inference over feature subsets, whose posterior mass favors low prediction error and low cross-environment variance. Our framework combines three key insights: First, we formulate feature selection as stability-aware posterior sampling. Here, causal invariance serves as a soft inductive bias rather than explicit causal discovery. Second, we train a diffusion model as a learned prior over plausible continuous selection masks, combined with a stability-aware likelihood that rewards invariance across environments. This diffusion prior captures structural dependencies among features and enables scalable exploration of the combinatorially large selection space. Third, we perform guided annealed Langevin sampling that combines the diffusion prior with the stability objective, which yields a tractable, uncertainty-aware posterior inference that avoids discrete optimization and produces robust feature selections. We evaluate CGDFS on open-source real-world datasets exhibiting distribution shifts. Across both classification and regression tasks, CGDFS consistently selects more stable and transferable feature subsets, which leads to improved out-of-distribution performance and greater selection robustness compared to sparsity-based, tree-based, and stability-selection baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。