用函数比较替代输出比较，提升各类学习任务表现
Functional Risk Minimization
- 以函数拟合数据点，损失基于函数间差异计算
- 在监督、无监督和强化学习中均优于传统方法
- 适合研究过参数模型泛化机制的学者
机器学习领域自1970年代以来已发生巨大变化，但其核心原则——经验风险最小化（ERM）仍保持不变。本文提出函数风险最小化（FRM），一种新框架，其中损失函数比较的是函数而非输出。该框架为每个数据点 $(x_i,y_i)$ 定义一个函数 $f_{θ_i}$，使其满足 $y_i = f_{θ_i}(x_i)$。FRM 能涵盖多种常见损失函数下的 ERM，同时更真实地建模噪声过程。此外，我们证明 FRM 为理解现代过参数化模型的泛化提供了新视角，其目标可被重新表述为寻找能拟合训练数据的最简单模型。
原文摘要 · Abstract (English)
The field of Machine Learning has changed significantly since the 1970s. However, its most basic principle, Empirical Risk Minimization (ERM), remains unchanged. We propose Functional Risk Minimization~(FRM), a general framework where losses compare functions rather than outputs. This results in better performance in supervised, unsupervised, and RL experiments. In the FRM paradigm, for each data point $(x_i,y_i)$ there is function $f_{θ_i}$ that fits it: $y_i = f_{θ_i}(x_i)$. This allows FRM to subsume ERM for many common loss functions and to capture more realistic noise processes. We also show that FRM provides an avenue towards understanding generalization in the modern over-parameterized regime, as its objective can be framed as finding the simplest model that fits the training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。