深度神经网络通过发现图像数据中的高阶关联结构实现有效泛化。
DNNs, Dataset Statistics, and Correlation Functions
- 将深度网络视为探测数据中多尺度关联模式的工具
- 成功分类模型必然捕捉到高阶相关函数
- 为突破传统学习理论的泛化悖论提供新视角
本文指出，数据集的内在结构在图像识别等任务中至关重要。具体而言，我们关注深度神经网络训练所依赖的实际数据集中相关性结构的本质与来源。我们认为，成功的深度神经网络本质上实现了凝聚态物理与材料科学中一种广泛使用的范式：关注介于基本原子/分子尺度与连续尺度之间的介观尺度相关结构。特别地，我们主张，在图像分类中表现良好的深度神经网络必须发现了高阶相关函数。众所周知，深度神经网络在明显违背标准统计学习理论的前提下仍能实现良好泛化。本文讨论这一观点对理解该现象的启示。
原文摘要 · Abstract (English)
This paper argues that dataset structure is important in image recognition tasks (among other tasks). Specifically, we focus on the nature and genesis of correlational structure in the actual datasets upon which DNNs are trained. We argue that DNNs are implementing a widespread methodology in condensed matter physics and materials science that focuses on mesoscale correlation structures that live between fundamental atomic/molecular scales and continuum scales. Specifically, we argue that DNNs that are successful in image classification must be discovering high order correlation functions. It is well-known that DNNs successfully generalize in apparent contravention of standard statistical learning theory. We consider the implications of our discussion for this puzzle.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。