arXiv:2606.11319cs.LGcond-mat.dis-nn2026-06

神经网络在输入90%被破坏时仍能准确分类,靠的是学习数据均值的原型机制。

Learning from almost nothing: How neural networks survive heavy input corruption

论文配图:Learning from almost nothing: How neural networks survive heavy input corruption
图 1 · 摘自论文原文
  • 用均值场方法分析无限宽网络,发现其决策基于类别均值
  • 即使输入90%被噪声破坏,模型准确率仍远超随机水平
  • 该机制适用于多种网络结构和激活函数,解释了鲁棒性来源

学习不完美数据是机器学习的核心课题,连接着鲁棒性与可学习性的根本问题。本文研究属性噪声:在标签保持完整的情况下,从被污染的输入中学习,这一设定相比标签噪声较少受到理论关注。我们考虑两种污染模型:加性噪声和替换噪声。通过在多层感知机(MLPs)上对污染分类数据集进行实验,发现神经网络具有强鲁棒性,在输入超过90%被破坏时仍能保持远高于随机水平的准确率——远超人类识别能力。为理解此现象,我们采用受均值场启发的方法,分析无限宽度网络在重污染条件下的行为,推导出分类结果的主导级决策规则:网络实现原型规则,即最近类均值(nearest-class-mean),将每个测试点分配给训练集中均值最接近的类别。该主导级决策规则在广泛的MLP架构中具有普适性,适用于任意深度、多种激活函数及噪声分布。实验表明,相同中心点机制也高度匹配有限宽度网络的行为,为学习在单个样本几乎无信号时仍能成功提供了可解释且可解析的理论依据。

原文摘要 · Abstract (English)

Learning from imperfect data is a central theme in machine learning, connecting practical questions of robustness to fundamental questions of learnability. Here we examine attribute noise: learning from corrupted inputs while keeping the labels intact, a setting that has received considerably less analytical attention than its label-noise counterpart. We consider two types of corruption models: additive noise and replacement noise. Through experiments with multi-layer perceptrons (MLPs) on corrupted classification datasets, we find that neural networks remain robust, maintaining well-above-chance accuracy even when inputs are >90% corrupted -- far beyond human recognition. To understand this robustness, we analyze infinite-width networks in the heavy-corruption regime using a mean-field-inspired approach and derive a leading-order decision rule for the classification outcome: the network implements a prototype rule, the nearest-class-mean, assigning each test point to the class whose training-set average it most closely resembles. This leading-order decision rule is universal across a broad range of MLP architectures, holding for any depth, as well as a wide class of activation functions and noise distributions. The same centroid mechanism closely matches finite-width network behavior in our experiments and provides an interpretable and analytically tractable account of why learning can succeed even when individual training examples carry almost no signal.

神经网络鲁棒性原型学习噪声容忍

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。