用排序一致性检测黑盒大模型是否被替换，防篡改且省查询。
Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test
- 基于输出排序的均匀性测试，不依赖模型参数或对数
- 在有限查询下，对量化、微调等替换均保持高检测力
- 适合关注模型安全与真实性的开发者和审核人员
随着API成为大语言模型的主要接口，用户通常面对缺乏透明度的黑盒系统。为降低成本或恶意篡改行为，提供商可能悄悄提供量化或微调后的模型变体，导致性能下降并威胁安全性。由于用户无法访问模型权重甚至输出对数，检测此类替换极为困难。为此，我们提出一种基于排序一致性的测试方法，可验证黑盒模型与本地部署的原始模型在行为上是否一致。该方法准确、查询高效，且避免可被探测的查询模式，对试图重定向或混合响应的对抗性提供者也具备鲁棒性。我们在多种威胁场景下评估该方法，包括量化、有害微调、越狱提示和完整模型替换，结果表明，在受限查询预算下，其统计功效显著优于已有方法。
原文摘要 · Abstract (English)
As API access becomes a primary interface to large language models (LLMs), users often interact with black-box systems that offer little transparency into the deployed model. To reduce costs or maliciously alter model behaviors, API providers may discreetly serve quantized or fine-tuned variants, which can degrade performance and compromise safety. Detecting such substitutions is difficult, as users lack access to model weights and, in most cases, even output logits. To tackle this problem, we propose a rank-based uniformity test that can verify the behavioral equality of a black-box LLM to a locally deployed authentic model. Our method is accurate, query-efficient, and avoids detectable query patterns, making it robust to adversarial providers that reroute or mix responses upon the detection of testing attempts. We evaluate the approach across diverse threat scenarios, including quantization, harmful fine-tuning, jailbreak prompts, and full model substitution, showing that it consistently achieves superior statistical power over prior methods under constrained query budgets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。