过参数化神经网络中，参数不确定性无法消除，即使函数已完全确定。
On the Epistemic Uncertainty of Overparametrized Neural Networks
- 从参数不可识别性出发，重新定义了认知不确定性
- 发现即使函数确定，参数仍存在显著不确定性
- 针对单隐层ReLU网络给出理论分析与实证验证
认知不确定性常被视为随数据增加可消除的不确定性，这一观点隐含参数可识别性的假设，并将认知不确定性等同于预测变异。然而，在过参数化神经网络中，由于对称性和冗余表示，模型参数通常不可识别。因此，即使底层函数已被完全确定，仍可能存在显著的参数不确定性。本文从不可识别性角度分析认知不确定性，刻画了离散与连续两类残余不确定性来源。以单隐层ReLU网络为研究对象，深入分析其后验结构，并通过实验验证了理论结论。
原文摘要 · Abstract (English)
Epistemic uncertainty is often viewed as a reducible uncertainty that vanishes with increasing data. This perspective implicitly assumes parameter identifiability and equates epistemic uncertainty with predictive variability. In overparametrized neural networks, however, model parameters are typically non-identifiable due to symmetries and redundant representations. As a consequence, substantial parameter uncertainty can persist even when the underlying function is fully identified. In this work, we analyze epistemic uncertainty through the lens of non-identifiability and characterize both discrete and continuous sources of residual uncertainty. Focusing on one-hidden-layer ReLU networks, we thoroughly analyze the resulting posterior structure and validate our theoretical insights through empirical studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。