数据粗粒化能提升模型性能，关键在于过滤不相关特征。
Data coarse graining can improve model performance
- 基于统计物理的粗粒化方法，按特征相关性筛选信息
- 高通滤波策略可降低预测风险，低通则相反
- 为数据增强提供理论支持，适合研究泛化机制者
损失性数据变换本质上会丢弃信息。然而在现代机器学习中，数据剪枝和有损数据增强反而能提升泛化能力。我们通过一个可解析的高维岭回归模型，研究了‘数据粗粒化’下的这一悖论。受统计物理重整化群启发，分析了根据特征对任务的相关性系统性丢弃特征的粗粒化方案。结果揭示预测风险随粗粒化程度呈非单调变化：高通方案（剔除信号弱的不相关特征）有助于泛化；而低通方案（整合信号强的相关特征）则有害。关键的是，在最优正则化下，这种非单调性是粗粒化本身所致，而非双下降效应的副产物。该框架为精心设计的数据增强为何有效提供了清晰解析解释——它剥离了不相关自由度，聚焦更预测性强的信号。结果凸显了由数据结构塑造的复杂、非单调的风险景观，并展示了统计物理思想在理解现代机器学习现象中的原则性价值。
原文摘要 · Abstract (English)
Lossy data transformations by definition lose information. Yet, in modern machine learning, methods like data pruning and lossy data augmentation can help improve generalization performance. We study this paradox using a solvable model of high-dimensional, ridge-regularized linear regression under 'data coarse graining.' Inspired by the renormalization group in statistical physics, we analyze coarse-graining schemes that systematically discard features based on their relevance to the learning task. Our results reveal a nonmonotonic dependence of the prediction risk on the degree of coarse graining. A 'high-pass' scheme--which filters out less relevant, lower-signal features--can help models generalize better. By contrast, a 'low-pass' scheme that integrates out more relevant, higher-signal features is purely detrimental. Crucially, using optimal regularization, we demonstrate that this nonmonotonicity is a distinct effect of data coarse graining and not an artifact of double descent. Our framework offers a clear, analytical explanation for why careful data augmentation works: it strips away less relevant degrees of freedom and isolates more predictive signals. Our results highlight a complex, nonmonotonic risk landscape shaped by the structure of the data, and illustrate how ideas from statistical physics provide a principled lens for understanding modern machine learning phenomena.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。