自动定位大模型推理中的无声错误，准确率超80%
Ekka: Automated Diagnosis of Silent Errors in LLM Inference
- 通过对比目标与参考框架的中间执行状态进行差分调试
- 在真实错误数据集上达到80%的单次诊断准确率
- 适合大模型服务框架开发者用于故障排查
大模型推理服务框架因快速迭代和复杂优化，易引入无声错误——输出质量悄然下降但无明确报错。由于高层症状与底层原因间存在巨大语义鸿沟，此类错误难以诊断。我们发现，借助语义正确的参考实现，可将诊断问题转化为差分调试任务。为此提出Ekka系统，通过系统化对齐并比较目标框架与参考框架的中间执行状态，定位根本原因。构建了来自主流服务框架的真实无声错误基准集，Ekka在该集上实现80%的pass@1诊断准确率和88%的pass@5准确率，优于现有最先进系统。此外，Ekka还发现了4个新的无声错误，均经开发团队确认。
原文摘要 · Abstract (English)
LLM serving frameworks are quickly evolving with a complex software stack and a vast number of optimizations. The rapid development process can introduce silent errors where output quality silently degrades without any explicit error signals. Diagnosing silent errors is notoriously difficult due to the substantial semantic gap between the high-level symptoms and the low-level root causes. We observe that diagnosis of silent errors can be effectively framed as a differential debugging problem by leveraging the existence of semantically correct reference implementations. We propose Ekka, an automated diagnosis system that identifies root causes by systematically aligning and comparing intermediate execution states between a target and a reference framework. We constructed a benchmark of real-world silent errors from popular serving frameworks, where Ekka shows 80% pass@1 diagnosis accuracy and 88% pass@5 diagnosis accuracy, outperforming state-of-the-art systems. Ekka also diagnoses 4 new silent errors from serving frameworks, all of which have been confirmed by the developers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。