受连分数启发的可解释神经网络,兼具高效训练与精准归因能力。
CoFrNets: Interpretable Neural Architecture Inspired by Continued Fractions
- 基于连分数结构设计新型神经网络,具可解析函数形式。
- 在合成与真实数据上实现媲美顶尖模型的准确率,且能估计特征贡献。
- 适合需要模型透明性与高精度的科研及工业场景。
近年来,针对神经网络的局部后验解释研究较多,但可解释神经架构的设计仍较匮乏。本文提出一种受连分数启发的新颖神经架构——CoFrNet,其具有数论中连分数的快速逼近等优良性质。我们证明该架构可通过特定函数形式实现高效训练与可解释性,并基于不同于传统无限宽度/深度路径的证明策略,建立其通用逼近能力。在非线性合成函数实验中,能精确建模并估计特征贡献,甚至捕捉高阶项。在涵盖表格、文本和图像的七个真实数据集上,其性能与可解释性模型及多层感知机相当或更优,部分接近先进模型水平。
原文摘要 · Abstract (English)
In recent years there has been a considerable amount of research on local post hoc explanations for neural networks. However, work on building interpretable neural architectures has been relatively sparse. In this paper, we present a novel neural architecture, CoFrNet, inspired by the form of continued fractions which are known to have many attractive properties in number theory, such as fast convergence of approximations to real numbers. We show that CoFrNets can be efficiently trained as well as interpreted leveraging their particular functional form. Moreover, we prove that such architectures are universal approximators based on a proof strategy that is different than the typical strategy used to prove universal approximation results for neural networks based on infinite width (or depth), which is likely to be of independent interest. We experiment on nonlinear synthetic functions and are able to accurately model as well as estimate feature attributions and even higher order terms in some cases, which is a testament to the representational power as well as interpretability of such architectures. To further showcase the power of CoFrNets, we experiment on seven real datasets spanning tabular, text and image modalities, and show that they are either comparable or significantly better than other interpretable models and multilayer perceptrons, sometimes approaching the accuracies of state-of-the-art models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。