提出一种新方法评估回归模型是否达到全局最优，无需真实数据生成机制。
A novel Information-Driven Strategy for Optimal Regression Assessment
- 基于输入与残差间的香农互信息构建评估框架
- 能有效检测回归模型是否达到全局最优
- 适合需要严格性能验证的科研与工业场景
在机器学习中，回归算法旨在最小化损失函数。评估方法需量化输入输出系统的真实响应与学习模型（学生）预测值之间的差异。由于缺乏真实数据生成机制，现有数据驱动评估方法无法保证全局最优性。本文提出信息教师（Information Teacher），一种新型数据驱动框架，可对回归算法进行形式化性能评估，确保全局最优性。该方法基于估计输入变量与残差之间的香农互信息（Shannon mutual information, MI），适用于广泛的加性噪声模型。数值实验表明，信息教师能够检测到全局最优性，其表现与不可达的真实模型下零估计误差一致，可作为真实评估损失的代理指标，为传统经验性能度量提供原则性替代方案。
原文摘要 · Abstract (English)
In Machine Learning (ML), a regression algorithm aims to minimize a loss function based on data. An assessment method in this context seeks to quantify the discrepancy between the optimal response for an input-output system and the estimate produced by a learned predictive model (the student). Evaluating the quality of a learned regressor remains challenging without access to the true data-generating mechanism, as no data-driven assessment method can ensure the achievability of global optimality. This work introduces the Information Teacher, a novel data-driven framework for evaluating regression algorithms with formal performance guarantees to assess global optimality. Our novel approach builds on estimating the Shannon mutual information (MI) between the input variables and the residuals and applies to a broad class of additive noise models. Through numerical experiments, we confirm that the Information Teacher is capable of detecting global optimality, which is aligned with the condition of zero estimation error with respect to the -- inaccessible, in practice -- true model, working as a surrogate measure of the ground truth assessment loss and offering a principled alternative to conventional empirical performance metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。