arXiv:2504.19811cs.CL2025-04被引 2

通过模型血缘关系预测大模型性能,提升预测准确率。

Can a Crow Hatch a Falcon? Lineage Matters in Predicting Large Language Model Performance

  • 引入血缘正则化矩阵分解,利用父模型关系建模
  • 在2934个模型上实现0.15~0.30更高的相关系数
  • 适合新模型冷启动时的性能预估与调参指导

在大规模语言模型(LLM)的训练或融合前,准确预测其性能可显著降低计算成本与开发周期。尽管已有方法如缩放定律考虑了参数量、训练样本等全局因素,却常忽略显式的模型血缘关系——即哪些模型源自哪些父模型。本文提出一种新型血缘正则化矩阵分解(LRMF)框架,通过图拉普拉斯正则项编码模型间的祖先关系。借助多跳父子连接,LRMF在实例级与基准级性能预测中均优于传统矩阵分解与协同过滤方法。大规模实验涵盖2,934个公开的Hugging Face模型及6个主要基准上的21,000+实例,结果显示引入血缘约束后,预测相关系数相比基线提升0.15至0.30。此外,LRMF有效缓解冷启动问题,即使数据极少,也能对新衍生或合并模型给出准确估计。该血缘引导策略为现代大模型开发中的超参数调优、数据选择与模型组合提供了资源高效的决策支持。

原文摘要 · Abstract (English)

Accurately forecasting the performance of Large Language Models (LLMs) before extensive fine-tuning or merging can substantially reduce both computational expense and development time. Although prior approaches like scaling laws account for global factors such as parameter size or training tokens, they often overlook explicit lineage relationships-i.e., which models are derived or merged from which parents. In this work, we propose a novel Lineage-Regularized Matrix Factorization (LRMF) framework that encodes ancestral ties among LLMs via a graph Laplacian regularizer. By leveraging multi-hop parent-child connections, LRMF consistently outperforms conventional matrix factorization and collaborative filtering methods in both instance-level and benchmark-level performance prediction. Our large-scale study includes 2,934 publicly available Hugging Face models and 21,000+ instances across 6 major benchmarks, showing that the introduction of lineage constraints yields up to 0.15-0.30 higher Pearson correlation coefficients with actual performance compared to baseline methods. Moreover, LRMF effectively addresses the cold-start problem, providing accurate estimates for newly derived or merged models even with minimal data. This lineage-guided strategy thus offers a resource-efficient way to inform hyperparameter tuning, data selection, and model combination in modern LLM development.

大模型预测血缘关系性能评估矩阵分解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。