显式定义模型与CUDA内核接口,自动发现大模型推理中的内存漏洞。
M2K: Making the Model-Kernel Interface Explicit for Reliable CUDA Kernel Verification
- 通过符号化追踪分离模型固定与用户可变参数,构建接口约束
- 在真实LLM系统中发现181个未知内存错误,仅9个误报
- 适用于大模型推理系统开发者和安全审计人员
大型语言模型(LLM)推理系统依赖CUDA内核进行核心GPU计算,但模型与内核之间的接口隐式且未明确定义。模型与内核独立演化,常对张量形状和输入大小做出不兼容假设,导致CUDA内核中出现细微的内存错误。这些错误可能引发推理服务崩溃、模型权重损坏,或被远程攻击者利用。现有技术要么带来巨大运行时开销,要么需要专用硬件,或无法处理动态张量形状与可变内核启动配置,致使CUDA内存错误长期未被解决。本文提出M2K,一个全自动框架,显式化模型-内核接口,并以此检测LLM推理系统中CUDA内核的内存错误。M2K包含两个组件:HFProbe在无GPU硬件环境下追踪模型执行,将内核参数分类为模型固定与用户可变,并生成捕捉接口的符号约束;cuKLEE则对CUDA内核进行符号执行,在接口约束下定位内存错误,将张量建模为互不重叠的内存区域,并对线程标识符进行符号处理,实现对数千线程的可扩展性。评估显示,M2K在真实LLM推理系统中发现了181个先前未知的漏洞,仅产生9个误报,证明其有效性。
原文摘要 · Abstract (English)
Large language model (LLM) inference systems rely on CUDA kernels for core GPU computations, yet the interface between models and kernels is implicit and poorly specified. Models and kernels evolve independently and often make incompatible assumptions about tensor shapes and input sizes, leading to subtle memory bugs in CUDA kernels. These bugs can crash inference services, corrupt model weights, or be exploited by remote adversaries. Existing techniques either incur prohibitive runtime overhead, require specialized hardware, or fail to handle dynamic tensor shapes and variable kernel launch configurations, leaving the CUDA memory bugs largely unaddressed. This paper presents M2K, a fully automated framework that makes the model-kernel interface explicit and leverages it to detect memory bugs in CUDA kernels used in LLM inference systems. M2K consists of two components. HFProbe traces model execution without GPU hardware, classifies kernel arguments into model-fixed and user-variable, and emits symbolic constraints that capture the interface. cuKLEE then performs symbolic execution on CUDA kernels to pinpoint memory bugs under the interface constraints, modeling tensors as disjoint memory regions and treating thread identifiers symbolically to scale to thousands of threads. In the evaluation, M2K discovers 181 previously unknown bugs in real LLM inference systems, while producing only nine false positives, demonstrating its effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。