提出可微的四边形损失,量化神经网络中特征交互强度,实现可调节的可解释性。
The Quadrilateral Loss: Additivity as a Measurable Behavior of Dense Neural Networks

- 设计四边形损失,通过二阶混合差分测量特征间交互作用。
- 小数据集上适度惩罚能同时提升准确率与模型可加性。
- 揭示正则化前后的交互保留差异,挑战事后交互重要性排序。
添加性模型通过禁止特征交互获得可解释性,而神经网络通常在结构上强制这一约束。本文提出四边形损失,一种可微的惩罚项,将加性视为可度量的行为:对训练样本对交换某一坐标后的二阶混合差分,当且仅当该坐标无交互时为零;对分段线性网络仍具信息量;其期望值等于干预式Shapley-GAM中每坐标的交互质量。该损失将加性变为可调节的旋钮——大多数学习到的交互几乎无成本移除,在小数据集上适度惩罚可同时提升精度与可加性;并成为在线可观测指标:各特征的投降曲线显示,正则化前的交互强度无法预测正则化后保留的交互,质疑了事后交互排名的有效性。我们以此工具对比多种精确加性路径:结构掩码、行为惩罚(可结晶为精确结构)、权重衰减、回代法、共享部分模型及袋装提升树桩。结果表明,行为约束优于权重空间约束,不同数据下排名反转,收敛路径在形状函数上达成一致。文中记录三种隐性失败模式,均源于将保证带入不满足前提的场景。
原文摘要 · Abstract (English)
Additive models buy interpretability by forbidding feature interactions, a constraint that neural instantiations enforce architecturally. We introduce the quadrilateral loss, a differentiable penalty that treats additivity as a measurable behavior instead: a second-order mixed difference on pairs of training points swapping one coordinate, which vanishes if and only if the coordinate carries no interaction, remains informative for piecewise-linear networks, and equals in expectation the per-coordinate interaction mass of the interventional Shapley-GAM. The loss turns additivity into a dial - most learned interactions prove removable almost for free, and on small datasets a moderate penalty improves accuracy and additivity simultaneously - and into an online observable: its per-feature surrender curves show, across seeds and datasets, that pre-regularization interaction magnitude barely predicts what a regularized model retains, undermining post-hoc interaction rankings. Against this instrument we compare routes to exact additivity, spanning structural masks, behavioral penalties (optionally crystallized into exact structure), weight decay, backfitting, the shared-section model, and bagged boosted stumps: constraining behavior before structure dominates weight-space constraints, rankings reverse between data regimes, and converging routes agree on the shape functions themselves. Three silent failure modes we document share one anatomy: guarantees imported into settings that quietly void their preconditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。