arXiv:2508.05463cs.LGcs.AI2025-08被引 4

任务越难,神经网络的权重结构越关键,越依赖符号拓扑而非具体数值。

Task complexity shapes internal representations and robustness in neural networks

  • 用五种无数据依赖的探测方法分析模型内部表示的拓扑结构。
  • 复杂任务模型在二值化后准确率跌至随机水平,简单任务则保持鲁棒。
  • 仅保留权重符号结构即可维持高精度,适合模型压缩与可解释性研究。

神经网络在各类任务中表现优异,但其内部表示如何受输入数据和问题复杂度影响仍不清晰。本文提出五种数据无关的探测方法——剪枝、二值化、噪声注入、符号翻转和双分图随机化——用于量化多层感知机(MLPs)在不同任务难度下的表征拓扑与鲁棒性。从网络科学视角,将MLP视为带符号的加权双分图。对比了在MNIST与Fashion-MNIST上的简单与复杂分类任务。结果显示,复杂任务模型在权重二值化后准确率降至随机水平,而简单任务模型仍保持稳定;在二值化后的复杂任务模型中,剪除低幅度边时出现明显的性能相变;适度噪声注入可提升准确率,类似最优小幅度权重符号翻转带来的随机共振效应;而仅保留符号结构的双分图随机化仍能维持高准确率。这些现象共同定义了一种模型与模态无关的任务复杂度度量:全精度与二值化/打乱网络性能之差。研究揭示了符号双分图拓扑在学习表示中的核心作用,并为与任务复杂度匹配的模型压缩与可解释性策略提供了实用思路。

原文摘要 · Abstract (English)

Neural networks excel across a wide range of tasks, yet remain black boxes. In particular, how their internal representations are shaped by the complexity of the input data and the problems they solve remains obscure. In this work, we introduce a suite of five data-agnostic probes-pruning, binarization, noise injection, sign flipping, and bipartite network randomization-to quantify how task difficulty influences the topology and robustness of representations in multilayer perceptrons (MLPs). MLPs are represented as signed, weighted bipartite graphs from a network science perspective. We contrast easy and hard classification tasks on the MNIST and Fashion-MNIST datasets. We show that binarizing weights in hard-task models collapses accuracy to chance, whereas easy-task models remain robust. We also find that pruning low-magnitude edges in binarized hard-task models reveals a sharp phase-transition in performance. Moreover, moderate noise injection can enhance accuracy, resembling a stochastic-resonance effect linked to optimal sign flips of small-magnitude weights. Finally, preserving only the sign structure-instead of precise weight magnitudes-through bipartite network randomizations suffices to maintain high accuracy. These phenomena define a model- and modality-agnostic measure of task complexity: the performance gap between full-precision and binarized or shuffled neural network performance. Our findings highlight the crucial role of signed bipartite topology in learned representations and suggest practical strategies for model compression and interpretability that align with task complexity.

神经网络可解释性模型压缩任务复杂度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。