研究ReLU网络在鲁棒记忆中的参数复杂度，发现鲁棒性越高所需参数越多。
The Cost of Robustness: Tighter Bounds on Parameter Complexity for Robust Memorization in ReLU Nets
- 通过分析鲁棒性比ρ=μ/ε，给出参数数量的紧致上下界。
- 当ρ较小时，鲁棒记忆与普通记忆所需参数量相当；ρ增大时参数量显著上升。
- 适用于关注神经网络鲁棒性与参数效率关系的研究者。
我们研究了ReLU网络在鲁棒记忆中的参数复杂度：即在不同标签点之间保持ε-分离的前提下，如何用最少参数实现对任意数据集的插值，并确保每个训练样本μ-邻域内的预测结果保持一致。我们建立了参数数量关于鲁棒性比ρ=μ/ε的上下界。与以往工作不同，本研究对ρ∈(0,1)的全范围进行了细粒度分析，获得了比现有结果更紧的上下界。研究发现，当ρ较小时，鲁棒记忆的参数复杂度与非鲁棒记忆相当；但随着ρ增加，所需参数数量随之增长。
原文摘要 · Abstract (English)
We study the parameter complexity of robust memorization for $\mathrm{ReLU}$ networks: the number of parameters required to interpolate any given dataset with $ε$-separation between differently labeled points, while ensuring predictions remain consistent within a $μ$-ball around each training sample. We establish upper and lower bounds on the parameter count as a function of the robustness ratio $ρ= μ/ ε$. Unlike prior work, we provide a fine-grained analysis across the entire range $ρ\in (0,1)$ and obtain tighter upper and lower bounds that improve upon existing results. Our findings reveal that the parameter complexity of robust memorization matches that of non-robust memorization when $ρ$ is small, but grows with increasing $ρ$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。