arXiv:2606.06335cs.LGcs.AI2026-06

用大模型和小模型融合预测,提升分布偏移下的性能估计准确性

Bridging Domain Expertise and Generalization for Performance Estimation

论文配图:Bridging Domain Expertise and Generalization for Performance Estimation
图 1 · 摘自论文原文
  • 结合基础模型与目标任务模型的预测分布,通过温度校准对齐
  • 融合加权生成参考标签,性能估计误差降低30%以上
  • 适合模型部署时评估未知数据表现,尤其适用于领域迁移场景

在分布偏移下进行性能估计的目标是预测模型在未标注测试集上的表现,而测试集分布与训练数据不同,需依赖不依赖真实标签的可靠指标。现有方法仅依赖目标模型输出,在分布偏移时其偏差被放大,导致与真实性能相关性下降。为此,我们提出融合参考对齐预测(FRAP),利用外部基础模型与目标模型的互补优势,构建更可靠的真值标签代理。FRAP通过温度缩放校准,最小化基础模型与目标模型预测分布间的差异,并基于置信度加权融合为精炼的参考分布,再以目标模型预测与该参考分布的吻合度进行性能估计。在多个数据集与模型架构上的大量实验表明,FRAP在分布偏移条件下显著优于代表性性能估计方法。

原文摘要 · Abstract (English)

Performance estimation under distribution shift aims to predict how a model behaves on an unlabeled test set whose distribution differs from the training data, a scenario that requires reliable indicators that can faithfully reflect model behavior without ground-truth labels. Existing approaches rely solely on the outputs of the given model whose biases are amplified once the distribution shifts, weakening the correlation with the true performance. Motivated by this limitation, we propose Fused Reference Alignment Prediction (FRAP), which leverages the complementary strengths of an external foundation model and the base model to construct a more reliable surrogate of the ground-truth labels. FRAP aligns the prediction distribution of the foundation model with that of the base model by applying temperature-scaled calibration that minimizes their divergence. The aligned predictions are fused through confidence-based weighting into a refined reference distribution that integrates robustness from the foundation model and domain-specific expertise from the base model, and performance estimation is obtained by measuring how closely the base model predictions agree with this reference. Extensive experiments across diverse datasets and architectures show that FRAP provides consistent and substantial improvements over representative performance-estimation methods under distribution shift.

性能估计分布偏移大模型融合模型评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。