用PAC理论证明：数据越多，模型性能越稳升。
Monotonic Learning in the PAC Framework: A New Perspective
- 基于PAC框架构建理论风险分布，分析学习算法性能变化
- 证明在有限假设空间或有限VC维下，算法性能随数据量单调提升
- 实验验证经典问题中泛化误差上界单调收敛至0，适合理论研究者
单调学习指随着训练数据增加，模型期望性能持续提升。然而近期研究挑战了这一传统认知，揭示机器学习泛化理论存在重要缺口。本文利用可能近似正确（PAC）学习理论，构建逼近学习算法实际性能的理论风险分布，并严格证明该分布随样本量增加呈现单调性。我们识别出两种确定性算法（基于经验风险最小化，ERM）具备单调性的场景：(1) 假设空间有限；(2) 假设空间具有有限VC维。在两个经典学习问题上的实验验证了该结论，显示算法泛化误差的理论风险上界随数据量单调收敛至0，证实了单调性可保证。
原文摘要 · Abstract (English)
Monotone learning describes learning processes in which expected performance consistently improves as the amount of training data increases. However, recent studies challenge this conventional wisdom, revealing significant gaps in the understanding of generalization in machine learning. Addressing these gaps is crucial for advancing the theoretical foundations of the field. In this work, we utilize Probably Approximately Correct (PAC) learning theory to construct a theoretical risk distribution that approximates a learning algorithm's actual performance. We rigorously prove that this theoretical distribution exhibits monotonicity as sample sizes increase. We identify two scenarios under which deterministic algorithms based on Empirical Risk Minimization (ERM) are monotone: (1) the hypothesis space is finite, or (2) the hypothesis space has finite VC-dimension. Experiments on two classical learning problems validate our findings by demonstrating that the monotonicity of the algorithms' generalization error is guaranteed, as its theoretical risk upper bound monotonically converges to 0.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。