arXiv:2409.17267cs.LGcs.AI2024-09ICLR被引 4

用方差最小化方法融合多个黑盒模型,提升预测准确性和鲁棒性。

Minimal Variance Model Aggregation: A principled, non-intrusive, and versatile integration of black box models

  • 通过最小化预测方差实现多模型线性融合,不依赖模型内部结构。
  • 在数据科学与偏微分方程任务中显著提升准确率和稳定性。
  • 适用于各类模型(如机器学习、数值求解器),无需修改原模型。

无论确定性或随机性,模型可视为近似特定目标量的函数。本文提出最小经验方差聚合(MEVA),一种数据驱动的多模型集成框架,通过利用各模型优势提升整体准确性。该非侵入式、模型无关的方法将参与模型视为黑箱,兼容多种方法输出,包括机器学习算法与传统数值求解器。我们主张逐点线性聚合,并提出两种优化策略:最小误差聚合(MEA)最小化预测误差,最小方差聚合(MVA)聚焦降低方差。理论证明,MVA比MEA更易从数据中稳健估计,使MEVA优于最小经验误差聚合(MEEA)。与直接插值目标值的MEEA不同,MEVA将聚合建模为误差估计问题,可使用任意主干学习范式完成。我们在数据科学和偏微分方程等任务中验证了框架的通用性与有效性,显著增强预测的鲁棒性与准确性。

原文摘要 · Abstract (English)

Whether deterministic or stochastic, models can be viewed as functions designed to approximate a specific quantity of interest. We introduce Minimal Empirical Variance Aggregation (MEVA), a data-driven framework that integrates predictions from various models, enhancing overall accuracy by leveraging the individual strengths of each. This non-intrusive, model-agnostic approach treats the contributing models as black boxes and accommodates outputs from diverse methodologies, including machine learning algorithms and traditional numerical solvers. We advocate for a point-wise linear aggregation process and consider two methods for optimizing this aggregate: Minimal Error Aggregation (MEA), which minimizes the prediction error, and Minimal Variance Aggregation (MVA), which focuses on reducing variance. We prove a theorem showing that MVA can be more robustly estimated from data than MEA, making MEVA superior to Minimal Empirical Error Aggregation (MEEA). Unlike MEEA, which interpolates target values directly, MEVA formulates aggregation as an error estimation problem, which can be performed using any backbone learning paradigm. We demonstrate the versatility and effectiveness of our framework across various applications, including data science and partial differential equations, illustrating its ability to significantly enhance both robustness and accuracy.

模型融合黑盒集成方差最小化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。