arXiv:2506.20927cs.LGcs.AI2025-06

用可学习的线性变换增强规则模型,让解释性更强且更简单。

Interpretable Representation Learning for Additive Rule Ensembles

  • 用可学习的稀疏线性变换代替固定阈值,生成更灵活的规则条件。
  • 在10个数据集上达到与顶尖方法相当的准确率,但模型复杂度显著降低。
  • 适合需要高可解释性又不想牺牲精度的场景,如医疗、金融决策。

小规模的符号规则集成提供可解释的预测模型。传统方法使用单一变量的简单阈值命题(如 $x \geq t$)构成规则,决策区域为轴对齐多面体。这种方法虽解释性强且可通过梯度提升高效学习,但依赖精心设计的表达性强且独立的输入特征;缺乏此类特征时,需增加规则数量和复杂度以提升精度,损害可解释性。本文提出扩展经典规则集成:引入可学习的稀疏线性变换输入变量的逻辑命题,形式为 $\mathbf{x}^\mathrm{T}\mathbf{w} \geq t$,其中 $\mathbf{w}$ 为可学习的稀疏权重向量,使决策区域变为具有斜面的一般多面体。我们提出基于迭代重加权逻辑回归的逐次贪心优化方法。实验表明,该方法在十个基准数据集上以更低的模型复杂度实现了与当前最优方法相当的测试风险。

原文摘要 · Abstract (English)

Small additive ensembles of symbolic rules offer interpretable prediction models. Traditionally, these ensembles use rule conditions based on conjunctions of simple threshold propositions $x \geq t$ on a single input variable $x$ and threshold $t$, resulting geometrically in axis-parallel polytopes as decision regions. While this form ensures a high degree of interpretability for individual rules and can be learned efficiently using the gradient boosting approach, it relies on having access to a curated set of expressive and ideally independent input features so that a small ensemble of axis-parallel regions can describe the target variable well. Absent such features, reaching sufficient accuracy requires increasing the number and complexity of individual rules, which diminishes the interpretability of the model. Here, we extend classical rule ensembles by introducing logical propositions with learnable sparse linear transformations of input variables, i.e., propositions of the form $\mathbf{x}^\mathrm{T}\mathbf{w} \geq t$, where $\mathbf{w}$ is a learnable sparse weight vector, enabling decision regions as general polytopes with oblique faces. We propose a learning method using sequential greedy optimization based on an iteratively reweighted formulation of logistic regression. Experimental results demonstrate that the proposed method efficiently constructs rule ensembles with the same test risk as state-of-the-art methods while significantly reducing model complexity across ten benchmark datasets.

规则学习可解释性模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。