检测大模型网关的隐性行为偏差，揭示服务与承诺不符的风险
Behavioral Consistency and Transparency Analysis on Large Language Model API Gateways
- 通过四维黑盒测试，审计网关响应内容、对话连贯性等行为
- 发现多数网关存在静默换模型、计费不准、延迟波动等问题
- 适合关注模型服务透明度的研究者和企业用户
第三方大语言模型API网关正成为多厂商模型的统一入口。然而，其内部路由、缓存与计费策略大多不公开，导致用户难以判断请求是否由宣称模型处理、响应是否忠实于上游接口，或账单是否符合公开定价。为此，我们提出GateScope——一种轻量级黑盒测量框架，用于评估商业大模型网关的行为一致性与运营透明度。该框架从响应内容分析、多轮对话表现、计费准确性与延迟特性四个维度，检测模型降级或切换、无声截断、计费错误及延迟不稳等关键异常。我们在10个真实商业网关上的测量显示，普遍存在预期与实际行为的显著差距，包括静默模型替换、记忆保持能力下降、定价声明偏离及平台间延迟稳定性差异显著。
原文摘要 · Abstract (English)
Third-party Large Language Model (LLM) API gateways are rapidly emerging as unified access points to models offered by multiple vendors. However, the internal routing, caching, and billing policies of these gateways are largely undisclosed, leaving users with limited visibility into whether requests are served by the advertised models, whether responses remain faithful to upstream APIs, or whether invoices accurately reflect public pricing policies. To address this gap, we introduce GateScope, a lightweight black-box measurement framework for evaluating behavioral consistency and operational transparency in commercial LLM gateways. GateScope is designed to detect key misbehaviors, including model downgrading or switching, silent truncation, billing inaccuracies, and instability in latency by auditing gateways along four critical dimensions: response content analysis, multi-turn conversation performance, billing accuracy, and latency characteristics. Our measurements across 10 real-world commercial LLM API gateways reveal frequent gaps between expected and actual behaviors, including silent model substitutions, degraded memory retention, deviations from announced pricing, and substantial variation in latency stability across platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。