一个采样方法同时高效估算多种特征重要性指标。
One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficiently
- 设计可统一适配多种概率值的采样框架,最大化样本复用。
- 理论证明并优化采样向量,实现所有值平均最优时间复杂度。
- 适用于需要对比多个特征重要性指标的场景,如数据估值与模型解释。
概率值(如贝塔谢林值、加权班扎夫值)在特征归因和数据估值中日益重要,但其精确计算常呈指数级开销,需依赖近似方法。已有研究表明不同概率值对下游性能影响显著,无统一最优选项,因此常需近似多个候选值再择优。尽管已有大量高效估计器研究,却均无法同时且高效地近似所有概率值。本文首次探索该目标:基于最大样本复用原则,提出参数化采样向量的‘一采样通用’框架,通过中间项转换生成任意概率值,无需放大标量。借助(ε, δ)-近似理论,我们推导出决定收敛速率的关键公式,并据此优化采样向量,得到:i) 平均意义下对所有概率值达到当前最优时间复杂度的‘一法通用’估计器;ii) 针对每类值最优调参的更快通用估计器。特别地,该通用估计器在贝塔谢林值(含经典谢林值)上实现理论与实证最优收敛速度。最后,我们揭示概率值与(正则化)数据模型中最小二乘回归之间的联系,表明该估计器可同时求解一类数据模型。
原文摘要 · Abstract (English)
The concept of probabilistic values, such as Beta Shapley values and weighted Banzhaf values, has gained recent attention in applications like feature attribution and data valuation. However, exact computation of these values is often exponentially expensive, necessitating approximation techniques. Prior research has shown that the choice of probabilistic values significantly impacts downstream performance, with no universally superior option. Consequently, one may have to approximate multiple candidates and select the best-performing one. Although there have been many efforts to develop efficient estimators, none are intended to approximate all probabilistic values both simultaneously and efficiently. In this work, we embark on the first exploration of achieving this goal. Adhering to the principle of maximum sample reuse, we propose a one-sample-fits-all framework parameterized by a sampling vector to approximate intermediate terms that can be converted to any probabilistic value without amplifying scalars. Leveraging the concept of $ (ε, δ) $-approximation, we theoretically identify a key formula that effectively determines the convergence rate of our framework. By optimizing the sampling vector using this formula, we obtain i) a one-for-all estimator that achieves the currently best time complexity for all probabilistic values on average, and ii) a faster generic estimator with the sampling vector optimally tuned for each probabilistic value. Particularly, our one-for-all estimator achieves the fastest convergence rate on Beta Shapley values, including the well-known Shapley value, both theoretically and empirically. Finally, we establish a connection between probabilistic values and the least square regression used in (regularized) datamodels, showing that our one-for-all estimator can solve a family of datamodels simultaneously.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。