用一次确定性排列取代多次随机排列,提升特征重要性计算的速度与稳定性。
One Permutation Is All You Need: Fast, Reliable Variable Importance and Model Stress-Testing
- 用单一最优排列替代多次随机排列,实现快速且确定性的特征重要性评估。
- 在近200个场景中验证,小样本、高维和低信噪比下表现更优。
- 新增系统性特征重要性,可检测模型对敏感属性的隐性依赖,适合公平性审计。
机器学习模型中可靠估计特征贡献对信任、透明度和合规性至关重要,尤其在模型为专有或黑箱时。尽管基于置换的方法是标准工具,但传统实现依赖重复随机置换,带来计算开销和随机不稳定性。本文表明,将多次随机置换替换为单次确定性最优置换,可在保留置换法核心原理的同时,实现非随机、更快、更稳定的效果。我们在近200个场景中验证该方法,涵盖真实世界家庭金融与信用风险应用,证明其在小样本、高维及低信噪比条件下具有更好的偏差-方差权衡和更高精度。最后,我们提出系统性特征重要性,一种用于模型压力测试的自然扩展,显式考虑特征相关性。该框架能透明量化扰动如何通过相关输入传播,揭示标准重要性度量所忽略的依赖关系。两个真实案例研究展示其如何用于审计模型对受保护属性(如性别或种族)的隐性依赖,使监管者与从业者能够以高效方式评估公平性与系统性风险。
原文摘要 · Abstract (English)
Reliable estimation of feature contributions in machine learning models is essential for trust, transparency and regulatory compliance, especially when models are proprietary or otherwise operate as black boxes. While permutation-based methods are a standard tool for this task, classical implementations rely on repeated random permutations, introducing computational overhead and stochastic instability. In this paper, we show that by replacing multiple random permutations with a single, deterministic, and optimal permutation, we achieve a method that retains the core principles of permutation-based importance while being non-random, faster, and more stable. We validate this approach across nearly 200 scenarios, including real-world household finance and credit risk applications, demonstrating improved bias-variance tradeoffs and accuracy in challenging regimes such as small sample sizes, high dimensionality, and low signal-to-noise ratios. Finally, we introduce Systemic Variable Importance, a natural extension designed for model stress-testing that explicitly accounts for feature correlations. This framework provides a transparent way to quantify how shocks or perturbations propagate through correlated inputs, revealing dependencies that standard variable importance measures miss. Two real-world case studies demonstrate how this metric can be used to audit models for hidden reliance on protected attributes (e.g., gender or race), enabling regulators and practitioners to assess fairness and systemic risk in a principled and computationally efficient manner.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。