Claude生成的Python测试质量不逊于人类编写，实证可靠。
The Quality of Claude AI-authored Python Tests Is Not Weaker Than Human-authored Tests
- 用真实项目代码生成测试，非孤立合成数据
- 多协议验证下AI测试与人工测试无显著差异
- 逐个测试评分，精准定位问题测试
我们评估了Claude AI编写的Python测试与两个知名开源项目Django和Pandas中人类编写的测试质量。每组数百个测试在相同评估协议下逐个评分。通过单侧非劣性检验发现，近期Claude模型（Sonnet/Opus 4.6及以后版本）编写的测试质量不低于两个真人测试集。本研究具备三个特点：(i) AI测试来自真实工具，而非孤立生成的合成测试，突破了以往所有相关研究的设定；(ii) 每个测试均通过三种独立故障注入协议和七维定性设计评分体系进行评分，实现方法互验；(iii) 测试评分以单个测试为单位，而非套件级别，可精确识别需改进的具体测试。
原文摘要 · Abstract (English)
We evaluate the quality of Claude AI-written Python tests against human-written Python tests from two established open-source projects Django and Pandas. Hundreds of tests per corpus are scored under one identical protocol. Using one-sided non-inferiority bounds, we find that the tests written by recent Claude models (Sonnet/Opus 4.6 and later) are no weaker than the two human-written corpora. In this study: (i) the AI-written corpus is tests from real tools, not synthetic tests generated in isolation against a fixed target, the setup used by every other AI-test-generation study we are aware of; (ii) every test is individually scored under three independent fault-injection protocols plus a seven-axis qualitative design rubric, allowing methods to cross-validate each other; (iii) tests are scored individually, rather than suite-level, identifying exactly which specific tests need attention.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。