arXiv:2604.00362cs.AIcs.LG2026-04被引 1

复现GPT-OSS-20B的基准成绩,突破官方未公开工具的限制。

In harmony with gpt-oss

论文配图:In harmony with gpt-oss
图 1 · 摘自论文原文
  • 逆向解析模型内建工具,发现其具备高置信度的工具调用先验。
  • 构建原生消息格式代理框架,避免接口转换损失信息。
  • 首次独立复现官方公布的60.4%、53.3%和91.7%关键指标。

目前无人能使用工具独立复现OpenAI发布的GPT-OSS-20B得分,因原始论文既未披露使用的工具,也未公开代理架构。本文通过逆向工程发现,即使在未提供工具定义的情况下,GPT-OSS仍能以高统计置信度调用训练分布中的工具——这反映的是强大的先验知识,而非幻觉。我们构建了原生harmony代理框架(https://github.com/borislavmavrin/harmonyagent.git),以模型原生格式编码消息,绕过有损的Chat Completions转换。由此实现首个独立复现:在SWE Verified HIGH上达到60.4%(官方公布60.7%),MEDIUM为53.3%(53.2%),使用工具时在AIME25上达91.7%(90.4%)。

原文摘要 · Abstract (English)

No one has independently reproduced OpenAI's published scores for gpt-oss-20b with tools, because the original paper discloses neither the tools nor the agent harness. We reverse-engineered the model's in-distribution tools: when prompted without tool definitions, gpt-oss still calls tools from its training distribution with high statistical confidence -- a strong prior, not a hallucination. We then built a native harmony agent harness (https://github.com/borislavmavrin/harmonyagent.git) that encodes messages in the model's native format, bypassing the lossy Chat Completions conversion. Together, these yield the first independent reproduction of OpenAI's published scores: 60.4% on SWE Verified HIGH (published 60.7%), 53.3% MEDIUM (53.2%), and 91.7% on AIME25 with tools (90.4%).

模型逆向工具调用基准复现代理框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。