即使加随机噪声,验证ReLU网络仍难,除非假设不成立。
Random Parameter Noise Does Not Make Exact ReLU Verification Easy
- 用高斯噪声扰动参数后,仍无法高效验证网络
- 在特定条件下,验证时间必然指数增长
- 适合研究验证复杂性的学者阅读
我们研究了在对抗性平滑模型下对ReLU神经网络进行精确验证的问题。每个网络权重和偏置独立地被高斯噪声扰动,截断于[-2,2]区间,并四舍五入到由输入位复杂度决定的精确二进制网格。在标准假设NP ⊈ BPP下,不存在对所有基础实例都有效的完全验证器,其期望运行时间在网络规模、位复杂度和逆噪声水平上为多项式。该结论在固定噪声水平σ⋆=2⁻¹¹时已成立:对于单隐藏层网络,输入在单位超立方体上,隐藏层维度不超过3,基础系数在[-1,1]内。证明结合了精确间隙嵌入与量化鲁棒性论证。对任意含m个子句的E3SAT公式Φ,每子句用四个ReLU构造出函数gΦ,满足max_{x∈[0,1]^n} gΦ(x)=(m−unsat(Φ))/3,且坐标阈值取整不会降低目标值。通过加权参数敏感性不等式和高斯集中性,证明所有系数聚合扰动后,验证间隙以至少1−e⁻ᵐ/⁸的概率保持线性于m。证明涉及截断、精确二进制舍入、输出层扰动、多项式位精度采样平滑高斯分布,以及将期望平滑运行时间转化为BPP算法。计算验证用于检查恒等式并展示广义与常数间隙的不同缩放行为,但仅为诊断而非复杂性定理证据。结果针对所提绝对噪声模型下的最坏情况基础网络,表明仅参数非退化不足以保证精确验证的普遍平滑多项式性。
原文摘要 · Abstract (English)
We study exact verification of ReLU networks in an adversarial smoothed model. Every network weight and bias is independently perturbed by Gaussian noise, clipped to $[-2,2]$, and rounded to the exact dyadic grid determined by the input bit complexity. We show that, under the standard assumption $\mathrm{NP}\not\subseteq\mathrm{BPP}$, there is no sound and complete verifier whose expected running time is polynomial in network size, bit complexity, and inverse noise level for every base instance. The conclusion already holds at the fixed noise level $σ_\star=2^{-11}$ for one-hidden-layer networks over a unit box, with hidden fan-in at most three and base coefficients in $[-1,1]$. The proof combines an exact gap embedding with a quantitative robustness argument. For every E3SAT formula $Φ$ with $m$ clauses, a four-ReLU-per-clause construction satisfies $\max_{x\in[0,1]^n} g_Φ(x)=(m-\operatorname{unsat}(Φ))/3$, and coordinatewise threshold rounding never decreases the objective. A weighted parameter-sensitivity inequality and Gaussian concentration then show that a verification gap linear in $m$ survives the aggregate perturbation of all coefficients with probability at least $1-e^{-m/8}$. The proof includes clipping, exact dyadic rounding, output-layer perturbations, polynomial-bit sampling of the rounded Gaussian law, and the conversion from expected smoothed running time to a BPP algorithm. Computational checks test the exact identity and illustrate the different scaling of extensive and constant gaps; they are diagnostics rather than evidence for the complexity theorem. The result concerns worst-case base networks in the stated absolute-noise model, but it shows that parameter nondegeneracy alone does not yield a universal smoothed-polynomial guarantee for exact verification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。