揭示神经网络电路发现的计算复杂性,为可解释性提供理论边界。
The Computational Complexity of Circuit Discovery for Inner Interpretability
- 用经典与参数化复杂性理论分析电路发现的各类查询问题。
- 多数查询属于难解或不可近似,对模型规模敏感且难以高效求解。
- 提出可转化路径与温和查询,帮助设计更可行的可解释性方法。
神经网络在机器学习、认知/脑科学及社会应用中的许多前景依赖于通过电路发现实现内部可解释性。然而,尽管启发式方法不断改进,其可扩展性与忠实性仍存疑,因我们对所解决问题的复杂性特征缺乏理解。本文采用经典与参数化计算复杂性理论研究电路发现:(1) 构建概念框架,从描述、解释、预测和控制角度分析电路查找查询;(2) 形式化一组机制解释查询,并提出分析框架;(3) 应用于多层感知机,确定多种实际相关查询变体与松弛的复杂度。结果揭示出严峻的复杂性景观:多数查询为难解,相对于模型/电路特征仍属固定参数难解,且在加法、乘法及概率近似方案下不可近似。为应对该困境,证明存在可将部分难题转化为更易处理形式的变换,并证明某些更温和的查询具有可处理性或固定参数可处理性。该框架使我们能界定可解释性查询的范围与局限,探索可行方案,并比较现有与未来架构的资源需求。
原文摘要 · Abstract (English)
Many proposed applications of neural networks in machine learning, cognitive/brain science, and society hinge on the feasibility of inner interpretability via circuit discovery. This calls for empirical and theoretical explorations of viable algorithmic options. Despite advances in the design and testing of heuristics, there are concerns about their scalability and faithfulness at a time when we lack understanding of the complexity properties of the problems they are deployed to solve. To address this, we study circuit discovery with classical and parameterized computational complexity theory: (1) we describe a conceptual scaffolding to reason about circuit finding queries in terms of affordances for description, explanation, prediction and control; (2) we formalize a comprehensive set of queries for mechanistic explanation, and propose a formal framework for their analysis; (3) we use it to settle the complexity of many query variants and relaxations of practical interest on multi-layer perceptrons. Our findings reveal a challenging complexity landscape. Many queries are intractable, remain fixed-parameter intractable relative to model/circuit features, and inapproximable under additive, multiplicative, and probabilistic approximation schemes. To navigate this landscape, we prove there exist transformations to tackle some of these hard problems with better-understood heuristics, and prove the tractability or fixed-parameter tractability of more modest queries which retain useful affordances. This framework allows us to understand the scope and limits of interpretability queries, explore viable options, and compare their resource demands on existing and future architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。