arXiv:2410.18268stat.MLcs.LG2024-10被引 4

提出新方法让模型选择更稳定,即使删掉一个数据点也不易变

Stabilizing black-box model selection with the inflated argmax

  • 用袋装+膨胀argmax组合提升选择稳定性
  • 在多种场景中选出的模型集合重叠率高且准确
  • 适合数据敏感、需可靠模型选择的研究者

模型选择是从候选模型中根据数据挑选最优的过程。例如LASSO和稀疏非线性动力学识别(SINDy)将模型选择转化为由训练数据决定的线性方程组的稀疏解问题。然而,在缺乏强假设的情况下,这类方法极不稳定:仅移除一个训练数据点就可能导致选择不同模型。本文提出一种新方法,结合袋装与“膨胀”argmax操作,实现理论保障的稳定性。该方法选取一组均能拟合数据的模型,且以高概率保证任意删除一个训练点后,新选出的模型集合仍与原集合有显著重叠。我们在四种场景验证:(a) 具有强相关协变量的模拟实验;(b) 基于洛特卡-沃尔泰拉模型的生态竞争分析;(c) 基于蛋白质组学细胞信号数据的图子集选择;(d) 无监督κ-均值聚类。结果表明,该方法在各类任务中均生成稳定、紧凑且准确的模型集合,优于多种基准方法。

原文摘要 · Abstract (English)

Model selection is the process of choosing from a class of candidate models given data. For instance, methods such as the LASSO and sparse identification of nonlinear dynamics (SINDy) formulate model selection as finding a sparse solution to a linear system of equations determined by training data. However, absent strong assumptions, such methods are highly unstable: if a single data point is removed from the training set, a different model may be selected. In this paper, we present a new approach to stabilizing model selection with theoretical stability guarantees that leverages a combination of bagging and an ''inflated'' argmax operation. Our method selects a small collection of models that all fit the data, and it is stable in that, with high probability, the removal of any training point will result in a collection of selected models that overlaps with the original collection. We illustrate this method in (a) a simulation in which strongly correlated covariates make standard LASSO model selection highly unstable, (b) a Lotka-Volterra model selection problem focused on identifying how competition in an ecosystem influences species' abundances, (c) a graph subset selection problem using cell-signaling data from proteomics, and (d) unsupervised $κ$-means clustering. In these settings, the proposed method yields stable, compact, and accurate collections of selected models, outperforming a variety of benchmarks.

模型选择稳定性袋装稀疏学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。