通过分析模型输出,可判断其运行在何种显卡和软件环境上。
Hardware and Software Platform Inference
- 基于输出数值模式识别显卡类型与软件栈
- 白盒测试下准确率83.9%至100%
- 适合关注模型服务真实性的买家或审计者
如今企业常购买大语言模型推理服务而非自建硬件,因前期投入巨大。但作为客户,无法验证服务商是否真用宣称的硬件(如NVIDIA H100)提供服务。有报告指出,服务商可能使用稍作修改的模型以适配廉价硬件,使客户支付高价却获得性能更低的服务。本文提出硬件与软件平台推断(HSPI),仅通过黑盒模型的输入输出行为,即可识别其底层GPU架构与软件配置。该方法利用不同GPU架构和编译器带来的计算差异,分析输出中的数值模式,构建分类框架。实验表明,在真实硬件部署中,白盒环境下对不同GPU的识别准确率达83.9%至100%,黑盒情况下性能仍为随机猜测的3倍以上。代码已开源。
原文摘要 · Abstract (English)
It is now a common business practice to buy access to large language model (LLM) inference rather than self-host, because of significant upfront hardware infrastructure and energy costs. However, as a buyer, there is no mechanism to verify the authenticity of the advertised service including the serving hardware platform, e.g. that it is actually being served using an NVIDIA H100. Furthermore, there are reports suggesting that model providers may deliver models that differ slightly from the advertised ones, often to make them run on less expensive hardware. That way, a client pays premium for a capable model access on more expensive hardware, yet ends up being served by a (potentially less capable) cheaper model on cheaper hardware. In this paper we introduce hardware and software platform inference (HSPI) -- a method for identifying the underlying GPU architecture and software stack of a (black-box) machine learning model solely based on its input-output behavior. Our method leverages the inherent differences of various GPU architectures and compilers to distinguish between different GPU types and software stacks. By analyzing the numerical patterns in the model's outputs, we propose a classification framework capable of accurately identifying the GPU used for model inference as well as the underlying software configuration. Our findings demonstrate the feasibility of inferring GPU type from black-box models. We evaluate HSPI against models served on different real hardware and find that in a white-box setting we can distinguish between different GPUs with between $83.9\%$ and $100\%$ accuracy. Even in a black-box setting we achieve results that are up to 3x higher than random guess accuracy. Our code is available at https://github.com/ChengZhang-98/HSPI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。