arXiv:2606.01400cs.CLcs.AI2026-06

用图算法选关键测试题,让大模型评测快4成还不丢准头。

Consistent and Distinctive: LLM Benchmark Efficiency via Maximum Independent Set Prompt Selection on Similarity Graphs

论文配图:Consistent and Distinctive: LLM Benchmark Efficiency via Maximum Independent Set Prompt Selection on Similarity Graphs
图 1 · 摘自论文原文
  • 把测试题建模成相似图,用最大独立集算法挑出不重复的代表性题目。
  • 在6个基准上平均减少35%测试题量,排名一致性超99%(肯德尔W≥0.90)。
  • 适合想高效评测大模型的研究者或部署团队,尤其关注成本与结果稳定性的场景。

对大语言模型进行全方位评估耗时且昂贵。本文提出一种基于图的提示选择框架,将每个基准视为相似性图——若提示在嵌入空间中的距离超过可配置阈值,则节点间相连——并应用最大独立集(MIS)算法选取多样且无冗余的子集。我们在六种嵌入模型、三种距离度量、六个百分位阈值及四个基准(GPQA、IFEval、MMLU-Pro、Omni-MATH)上评估了四种MIS求解器(CPLEX、GREEDY、Online-MIS、ReduMIS),覆盖66个大模型。核心假设——不同随机种子下重复选择仍能保持一致的模型排名,且可能不同于全基准基线——得到强有力验证:99.2%的随机配置中肯德尔等级相关系数W≥0.90(均值W=0.997±0.008),高百分位阈值下平均提示量减少25%–48%。与全基准排名差异显著(ρ<0.95)的情况仅占15.95%,主要集中在低阈值(p₁₀–p₂₀)和特定基准(GPQA、IFEval),表明过密图是主要失败模式。

原文摘要 · Abstract (English)

Evaluating large language models (LLMs) across comprehensive benchmarks is expensive and time-consuming. We propose a graph-based prompt selection framework that models each benchmark as a similarity graph -- nodes are prompts connected if their embedding-space distance falls above a configurable threshold -- and applies Maximum Independent Set (MIS) algorithms to select a maximally diverse, non-redundant subset. We evaluate four MIS solvers (CPLEX, GREEDY, Online-MIS, ReduMIS) across six embedding models, three distance measures, six percentile thresholds, and four benchmarks (GPQA, IFEval, MMLU-Pro, Omni-MATH) covering 66 LLMs. Our central hypothesis -- that repeated selection under different random seeds yields consistent LLM rankings that may also differ from the full-benchmark baseline -- is strongly confirmed: Kendall's $W \geq 0.90$ in 99.2\% of stochastic configurations (mean $W = 0.997 \pm 0.008$), while at higher percentile thresholds selected subsets achieve 25--48\% prompt reduction on average. Ranking divergence from the full benchmark ($ρ< 0.95$) occurs in only 15.95\% of configurations, concentrated at low thresholds ($p_{10}$--$p_{20}$) and benchmarks (GPQA, IFEval), identifying overly dense graphs as the primary failure mode.

模型评测图算法效率优化提示筛选

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。