用验证优先策略让不同模型协作更可靠,提升代码与问答准确率。
Agreement Before Diversity: Verification-First Complementarity for Heterogeneous Language-Model Coordination
- 先验证再引入多样性:锚定答案需两份可信样本支持才保留
- 在LiveCodeBench上达59.43%准确率,超越单模型和传统集成方法
- 无需标签、可审计,适合对结果可靠性要求高的场景
异构语言模型集成拓展了候选回答空间,但缺乏判断何时应替换已有答案的合理标准。本文提出验证优先的互补机制(ABD):将候选空间与替换权限解耦,使后者成为可显式审计的对象。锚定答案仅在两个额外可信样本在固定等价关系下支持时保留,否则由异构合成替代。该门控机制满足两个精确恒等式:其一,相对于无条件合成的准确率差距由一致覆盖度与锚点在保护子集上的优势共同决定;其二,相对于从不合成,差距体现为授权恢复与授权破坏的对比。两者均不依赖独立性或校准置信度,预期推理开销约为8减去5倍覆盖次数。盲测下,ABD在LiveCodeBench-v6上达59.43%(n=175),优于Single9(52.57%)与HAC(52.00%);在未触碰的GPQA-Diamond上达75.00%(控制组72.78%,n=180)。这些恒等式可定位所有聚合差异至可枚举的保护层:LiveCodeBench中3个保护案例无矛盾项,覆盖度预示门控贡献上限为1.71分;GPQA-Diamond中13对8个矛盾案例;冻结锚点扰动下71例中0矛盾。多样性提供潜力,验证结构赋予权威。
原文摘要 · Abstract (English)
Heterogeneous language-model ensembles expand the space of candidate responses, yet they lack a principled criterion for when a newly generated answer should supersede an already supported one. We decouple candidate headroom from replacement authority, rendering the latter as an explicit, auditable object. Our proposed method, Agreement-Before-Diversity (ABD), is a frozen, label-free decision rule: an anchor answer is retained if two additional trusted samples corroborate it under a fixed equivalence relation; otherwise, it is replaced by a heterogeneous synthesis. For this gating mechanism, we prove two exact identities. The first shows that the accuracy gap relative to unconditional synthesis is determined jointly by the agreement coverage and the anchor's advantage on the protected subset. The second shows that the gap relative to never synthesizing reflects a contrast between authorized recovery and authorized destruction. Neither identity assumes independence or calibrated confidence, and the expected inference cost is approximately eight minus five times the coverage in number of calls. Under blind, exact-ID evaluation, ABD achieves 59.43% on the complete LiveCodeBench-v6 (vs. 52.57% for Single9 and 52.00% for HAC; n = 175) and 75.00% on an untouched GPQA-Diamond split (both controls at 72.78%; n = 180). Furthermore, these identities localize every aggregate difference to an enumerable protected stratum: no discordant items occur among the 3 protected cases on LiveCodeBench, where coverage bounds the gate's contribution to 1.71 points a priori; 13 versus 8 discordant cases among 132 on GPQA-Diamond; and 12 versus 0 among 71 under a frozen anchor perturbation. Diversity supplies potential; verification structure supplies authority.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。