提出可自动生成逻辑解释的神经网络,提升表格分类可解释性。
Neural Reasoning Networks: Efficient Interpretable Neural Networks With Automatic Textual Explanations
- 用逻辑神经元构建可扩展架构,结合梯度下降与强化学习优化结构
- 在22个数据集上达到接近主流模型的准确率,参数量减少100倍以上
- 生成更短且更准的特征重要性解释,适合需透明决策的场景
近年来,神经网络在各类任务中广泛应用,但缺乏可解释性仍制约其在公平性、安全性及合规性要求高的场景落地。本文聚焦表格数据分类任务,提出一种新型神经符号架构——神经推理网络(NRN),具备可扩展性并能自动生成逻辑严谨的文本解释。NRN由逻辑神经元构成,实现实值逻辑运算。训练算法R-NRN采用梯度下降优化权重,同时通过基于博弈的优化自动学习网络结构,全部集成于PyTorch扩展(https://github.com/IBM/torchlogic),支持GPU加速与批量训练。在22个开源表格分类数据集上的评估显示,其性能(以ROC AUC衡量)优于多层感知机(MLP),与随机森林、XGBoost、梯度提升树等先进方法相当,同时训练速度提升43%,平均参数量减少超过两个数量级。此外,R-NRN生成的解释更简洁,特征重要性评分更准确。
原文摘要 · Abstract (English)
Recent advances in machine learning have led to a surge in adoption of neural networks for various tasks, but lack of interpretability remains an issue for many others in which an understanding of the features influencing the prediction is necessary to ensure fairness, safety, and legal compliance. In this paper we consider one class of such tasks, tabular dataset classification, and propose a novel neuro-symbolic architecture, Neural Reasoning Networks (NRN), that is scalable and generates logically sound textual explanations for its predictions. NRNs are connected layers of logical neurons which implement a form of real valued logic. A training algorithm (R-NRN) learns the weights of the network as usual using gradient descent optimization with backprop, but also learns the network structure itself using a bandit-based optimization. Both are implemented in an extension to PyTorch (https://github.com/IBM/torchlogic) that takes full advantage of GPU scaling and batched training. Evaluation on a diverse set of 22 open-source datasets for tabular classification demonstrates performance (measured by ROC AUC) which improves over multi-layer perceptron (MLP) and is statistically similar to other state-of-the-art approaches such as Random Forest, XGBoost and Gradient Boosted Trees, while offering 43% faster training and a more than 2 orders of magnitude reduction in the number of parameters required, on average. Furthermore, R-NRN explanations are shorter than the compared approaches while producing more accurate feature importance scores.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。