通过智能采样破解黑箱神经网络，实现超百万参数精准重建
Sequencing the Neurome: Towards Scalable Exact Parameter Reconstruction of Black-Box Neural Networks
- 利用随机初始化与一阶优化的先验，缩小参数搜索空间
- 生成高信息量查询样本，高效解析神经元间非线性关系
- 成功复现超150万参数、7层深度网络，误差低于0.0001
仅通过查询访问推断神经网络的精确参数是一个NP难问题，现有实用算法极少。此类方法对安全、验证、可解释性及生物网络理解具有重大意义。核心挑战在于庞大的参数空间和神经元间的复杂非线性关系。本文提出两个关键洞察：首先，几乎所有实际使用的网络均通过随机初始化与一阶优化生成，这一归纳偏置显著缩小了实际参数空间；其次，提出一种新型查询生成算法，可生成最大信息量样本，从而高效解开非线性关系。我们成功实现了包含超过150万参数的隐藏网络重建，以及7层深度网络的重建，为迄今最大、最深的精确重建，最大参数差异小于0.0001，并在多种架构、数据集与训练流程下展示出鲁棒性与可扩展性。
原文摘要 · Abstract (English)
Inferring the exact parameters of a neural network with only query access is an NP-Hard problem, with few practical existing algorithms. Solutions would have major implications for security, verification, interpretability, and understanding biological networks. The key challenges are the massive parameter space, and complex non-linear relationships between neurons. We resolve these challenges using two insights. First, we observe that almost all networks used in practice are produced by random initialization and first order optimization, an inductive bias that drastically reduces the practical parameter space. Second, we present a novel query generation algorithm that produces maximally informative samples, letting us untangle the non-linear relationships efficiently. We demonstrate reconstruction of a hidden network containing over 1.5 million parameters, and of one 7 layers deep, the largest and deepest reconstructions to date, with max parameter difference less than 0.0001, and illustrate robustness and scalability across a variety of architectures, datasets, and training procedures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。