arXiv:2607.04360stat.MLcs.LG2026-07

让多个生成模型自动协作,根据输入动态加权,提升生成效果。

Optimal Mixture-of-Experts Model Averaging for Conditional Generative Models

  • 用样本差异度衡量生成模型差异,静态加权固定组合,动态加权随输入调整。
  • 在表格、图像、文本数据上均优于基线,动态加权方法显著提升生成质量。
  • 适合多模型协同场景,尤其适用于无法直接计算密度的生成任务。

条件生成模型已成为从目标条件分布采样的强大工具,在科学与应用领域推动了显著进展。随着模型数量增加,不同生成器的表现常随任务、数据或输入条件变化。本文提出一种面向条件生成模型的最优模型平均框架,允许候选生成器仅通过条件样本接入,无需可计算密度。首先基于样本最大均值差异(MMD)构建静态模型平均方法(StaticMA),分配固定权重;进一步提出MoEMA(专家混合模型平均),通过软最大化神经网络门控参数化协变量依赖的权重。本文建立了所提方法的样本内与样本外渐近最优性,并在正则条件下证明了自适应权重函数的一致性。该框架可直接应用于欧氏响应,亦可通过固定表示映射扩展至非结构化数据。在涵盖表格、图像与文本模态的广泛模拟与真实数据研究中,MoEMA普遍优于现有基线,验证了方法的有效性。

原文摘要 · Abstract (English)

Conditional generative models have emerged as powerful tools for sampling from target conditional distributions, driving substantial advances across a wide range of scientific and applied domains. As these models proliferate, practitioners often face multiple plausible generators whose performance can vary with the task, data, or input condition. We propose an optimal model averaging framework for conditional generative models, allowing candidate generators to be combined even when they are accessible only through conditional samples without tractable densities. Specifically, we use a sample-based maximum mean discrepancy between conditional distributions, which first leads to a static model averaging method, StaticMA, assigning fixed weights to different candidates. In addition, we develop MoEMA (mixture-of-experts model averaging), an input-adaptive method that parameterizes covariate-dependent weights through a softmax neural-network gate. We establish in-sample and out-of-sample asymptotic optimality for the proposed methods, together with consistency of the estimated adaptive weight function under regularity conditions. The framework applies directly to Euclidean responses and extends to unstructured data by combining our formulation with fixed representation maps. Across a broad set of simulations and real-data studies spanning tabular, image, and text modalities, MoEMA generally improves over competing baselines, demonstrating the effectiveness of our proposed methods.

生成模型模型平均条件生成动态加权

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。