arXiv:2608.01502cs.LGcs.CV2026-08

提出随机搜索方法,让高维特征选择更高效且保留特征协同效应。

Stochastic Sequential Search in Very-High-Dimensional Feature Selection

  • 用温度控制采样替代全量遍历,每步只评估固定数量候选特征。
  • 在500维madelon上保持97%性能,仅需四分之一计算量;10105维reuters上超越现有方法。
  • 适合大规模高维特征选择任务,尤其当特征间存在协同效应时。

序列子集搜索(如前向选择带浮点回溯)虽是特征选择的高质量基准,但每步需遍历全部剩余特征,难以应对极高维问题。此时仅单特征排序可行,却忽略特征间交互。本文提出预算约束的采样步骤算子对,以固定次数评估替代全量扫描;候选特征通过依赖感知的在线统计,经温度调节softmax采样生成,辅以均匀探索下限。每步成本与维度无关。替换算子后,任意序列方法可转化为其随机版本,定义为随机序列搜索(SSS)家族,本文研究sSFFS。在500维madelon上,sSFFS在各子集大小下均保持至少97%的全量SFFS准则值,评估量仅为四分之一;均匀采样在同一预算下因无法捕捉协同特征而失效。在5000维gisette上,远超全量SFFS能力范围,sSFFS在匹配预算下超越DAF和BIF排名的饱和准则值;留出验证显示,500训练样本下,性能瓶颈由准则而非搜索本身决定。在10105维reuters上,基于可信多项式过滤准则,sSFFS在所有子集大小下均优于BIF和DAF的搜索目标及留出准确率,单核运行约两分钟。论文附有可验证的独立实现。

原文摘要 · Abstract (English)

Sequential subset search -- forward selection with floating backtracking and its descendants -- remains the quality reference in feature selection, but every member of the family sweeps the full pool of remaining candidate features at each step, which excludes it from very-high-dimensional problems; there, only individual-feature ranking remains practical, and it models feature interplay weakly or not at all. We introduce a budgeted sampled step operator pair that replaces the full sweeps by a fixed number of candidate evaluations per step. Candidates are drawn by temperature-controlled softmax sampling from dependency-aware per-feature statistics learned online from every criterion evaluation the search performs, guarded by a uniform exploration floor; per-step cost becomes independent of dimensionality. Substituting the operators turns any sequential method into its stochastic counterpart, defining the Stochastic Sequential Search (SSS) family; we study the stochastic counterpart of floating search, sSFFS. On 500-dimensional madelon, sSFFS retains at least 97% of the full-SFFS criterion value at every subset size at about a quarter of its evaluations, while uniform sampling at the same budget collapses on madelon's synergistic features. On 5,000-dimensional gisette, far beyond full-SFFS reach, sSFFS exceeds the saturated criterion level of DAF and BIF ranking at matched budgets; holdout validation shows that at 500 training samples the binding constraint beyond the sequential frontier becomes the criterion, not the search. On 10,105-dimensional reuters, under a trustworthy multinomial filter criterion, sSFFS dominates BIF and DAF on the search objective and on holdout accuracy at every subset size, in about two minutes of single-core evaluation work. A verified standalone implementation accompanies the paper.

特征选择高维数据随机搜索机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。