评测大模型在信息学奥赛中的表现，发现顶尖模型仍不如人类高手。
LiveOIBench: Can Large Language Models Outperform Human Contestants in Informatics Olympiads?
- 构建403道专家设计的编程题库，每题平均60个测试用例。
- GPT-5仅达人类前18.24%，开源模型最高仅达前40%。
- 全离线评估系统，支持与顶尖人类选手直接对比。
竞赛编程问题因其复杂性和可验证性，正被广泛用于评估大语言模型的编码能力。然而现有基准存在难题不足、测试用例覆盖不全、依赖在线平台API导致访问受限等问题。为此，我们提出LiveOIBench，一个包含403道由专家精心设计的编程题的大规模基准，题源来自2023至2025年间14个信息学奥林匹克竞赛的72场赛事，每题平均含60个官方测试用例。该基准具备四大特点：（1）专家设计的任务与详尽的子任务评分标准及丰富测试用例；（2）可直接与精英人类选手进行对比；（3）持续更新以降低数据污染风险；（4）完全离线、可复现的评估系统。对34种主流通用及推理型LLM的评测显示，GPT-5达到81.76百分位，仍低于顶尖人类选手；而在开源模型中，GPT-OSS-120B仅达60百分位。推理轨迹分析表明，强推理模型更注重精准的问题分析而非过度探索。跨发布日期、任务熟悉度与代码相似性分析均未发现显著数据污染证据。排行榜、代码与数据已公开于：https://liveoibench.github.io/。
原文摘要 · Abstract (English)
Competitive programming problems are increasingly used to evaluate the coding capabilities of large language models (LLMs) due to their complexity and ease of verification. Yet, current coding benchmarks face limitations such as a lack of exceptionally challenging problems, insufficient test case coverage, and reliance on online platform APIs that limit accessibility. To address these issues, we introduce LiveOIBench, a large-scale competitive programming benchmark featuring 403 expert-curated problems, averaging 60 official test cases each, drawn from 72 contests across 14 Informatics Olympiads held between 2023 and 2025. LiveOIBench has four key features: (1) expert-designed tasks with detailed subtask rubrics and extensive test cases; (2) direct comparison to elite human contestants; (3) continuous updates to reduce contamination risk; and (4) a fully offline, reproducible evaluation system. Benchmarking 34 popular general-purpose and reasoning LLMs, we find that GPT-5 achieves an 81.76th percentile, still falling short of top human contestants, while among the open-weight models, GPT-OSS-120B reaches only the 60th percentile. Reasoning-trace analyses indicate that robust reasoning models prioritize precise problem analysis over excessive exploration. Finally, analyses across release dates, task familiarity, and code similarity find minimal evidence of data contamination in our benchmark. Our leaderboard, code, and data are available at: https://liveoibench.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。