评测大模型对代码浮点错误的识别能力，发现顶尖模型准确率超88%。
Benchmarking Large Language Models on Floating-Point Error Classification
- 将浮点错误检测建模为多标签分类任务，用F1分数评估性能。
- 最新模型整体F1超0.88，但对隐蔽错误如舍入误差识别率仅60%。
- 适合关注代码安全与自动化调试的研究者和开发者参考。
本文研究大型语言模型（LLMs）在软件代码中静态检测和分类浮点错误的能力。我们提出了InterFLOPBench基准，包含90个C核函数和1130个测试样本，用于评估14个LLMs在六类浮点错误（舍入、比较、除零、溢出、下溢和NaN）上的表现。评估框架将浮点错误检测视为多标签分类问题，并采用F1分数作为衡量指标。结果表明，最新模型（Qwen 3 32b、Gemini 2.5 Flash、Phi 4 Reasoning、DeepSeek R1T2、gpt-oss 20b 和 120b）的整体F1分数超过0.88。不同错误类别表现差异明显：除零等显式操作平均F1为0.8479，而下溢（0.6059）和舍入误差（0.6164）等隐性现象识别率较低。
原文摘要 · Abstract (English)
This paper investigates the capability of Large Language Models (LLMs) to detect and classify floating-point errors statically in software code. We introduce InterFLOPBench, a benchmark of 90 C kernels with 1 130 test samples designed to evaluate LLMs across six categories of floating-point error: cancellation, comparison, division by zero, overflow, underflow and NaN, compared across 14 LLMs. The evaluation framework treats floating-point error detection as a multi-label classification problem and employs the F1-score metric to measure performance. Results demonstrate that latest models (Qwen 3 32b, Gemini 2.5 Flash, Phi 4 Reasoning, DeepSeek R1T2, and gpt-oss 20b and 120b) achieve a performance greater than 0.88 overall F1-score. Performance varies between error categories, between explicit operations such as division by zero (Average F1-score: 0.8479) and more subtle numerical phenomena such as underflow (Average F1-score: 0.6059) and cancellation (Average F1-score: 0.6164).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。