提出可检测生成核函数错误的严格验证器,发现现有测试大量误判。
A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family
- 构建12个无容差约束的校验门,严格验证核函数正确性
- 2638个生成核中39.5%存在不可容忍错误,62.1%至少一处违规
- 适用于验证生成式系统输出质量,尤其适合高可靠性场景
以语言模型生成的GPU核函数虽报告高正确率,但其评估仅依赖单一松散测试:在固定形状下对少数随机输入运行并接受输出接近参考值的结果。此类测试无法发现隐性错误,如返回非正常数(NaN/无穷)或随运行波动、形状变化即失效,或在fp16中累积而参考值为fp32。为此,我们构建了十二个对抗性校验门组成的合同级验证器,每个均为核函数必须满足的性质,其中部分为容差零容忍,无法用阈值解释失败。验证器审计了公开系统自身测试通过的2,638个生成核,发现39.5%存在不可容忍错误,62.1%至少有一项违反。标准测试接受的1,487个核中,验证器拒绝了其中多数,仅有14个反向情况。通过七次正向控制实验、阈值校准扫描、与参考基准代码98.5%一致性及分层人工审计,验证结果可靠。同时,该验证器确认了首个原生Blackwell tcgen05训练后向针对门控线性递归(GDN)家族的核函数正确性,包括领域仍依赖回退方案的逆状态阶段。在双精度基准下独立验证其正确性,并以此训练该家族五个成员。目前核函数生成所依赖的正确性信号远弱于数据呈现,采用一组容差零容忍的合约可弥合大部分差距。
原文摘要 · Abstract (English)
Systems that generate GPU kernels with language models report high correctness rates. Those rates come from a single loose test: run the kernel on a few random inputs at one fixed shape and accept it if the output is close to a reference. A kernel can pass that test and still be silently wrong. It can return an ordinary number where the true answer is a NaN or an infinity, differ from run to run, break when the shape changes, or accumulate in fp16 where the reference keeps an fp32 total. We build the instrument that checks correctness properly: a contract-grade verifier of twelve adversarial gates, each a property a correct kernel must satisfy, several of them tolerance-free, so no choice of threshold can explain a failure away. Aimed outward, the verifier audits 2,638 machine-generated kernels that a public system's own harness had already accepted as correct. It finds 39.5% broken beyond any tolerance argument and 62.1% carrying at least one violation. The field's standard test accepts 1,487 kernels the verifier rejects, against only 14 the other way. We defend the finding four independent ways: a 7/7 positive control, a threshold-calibration sweep, 98.5% agreement with the reference benchmark's own correctness code, and a stratified hand-audit. Aimed inward, the verifier judges a kernel of our own: the first native Blackwell tcgen05 training backward for the gated-linear-recurrence (GDN) family, including the reverse-state stage the field still runs on a fallback. We establish its correctness independently, against a double-precision oracle, and train five family members through it. The correctness signal behind reported progress in kernel generation is far weaker than the numbers suggest, and a set of tolerance-free contracts would close most of the gap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。