揭露大模型服务隐藏操作,呼吁建立可审计的计费机制
Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Services
- 提出用户无法看见的内部操作可能被虚报
- 发现计费存在数量虚增和质量下降两大风险
- 适合关注模型服务透明度的研究者与政策制定者
现代大型语言模型(LLM)服务依赖复杂的多步推理和多代理协作等抽象操作生成高质量输出。尽管用户按调用次数和令牌消耗付费,但这些内部步骤通常不可见。我们称此类系统为商业不透明语言模型服务(COLS)。本文指出COLS中的问责挑战:用户为无法观测、验证或质疑的操作付费。我们形式化了两个核心风险:数量虚增(令牌与调用次数被人为抬高)与质量下降(服务商可能悄悄替换为低成本模型或工具)。解决这些风险需采用承诺式、预测式、行为式和签名式等多种审计策略。我们进一步探讨水印技术和可信执行环境等互补机制在不泄露专有信息前提下提升可验证性的潜力。最后,提出一个模块化的三层审计框架,涵盖执行层、安全日志层与用户可审计层,实现信任验证而无需暴露内部机密。旨在推动关于商业LLM服务透明性、可审计性与问责性的研究与政策发展。
原文摘要 · Abstract (English)
Modern large language model (LLM) services increasingly rely on complex, often abstract operations, such as multi-step reasoning and multi-agent collaboration, to generate high-quality outputs. While users are billed based on token consumption and API usage, these internal steps are typically not visible. We refer to such systems as Commercial Opaque LLM Services (COLS). This position paper highlights emerging accountability challenges in COLS: users are billed for operations they cannot observe, verify, or contest. We formalize two key risks: \textit{quantity inflation}, where token and call counts may be artificially inflated, and \textit{quality downgrade}, where providers might quietly substitute lower-cost models or tools. Addressing these risks requires a diverse set of auditing strategies, including commitment-based, predictive, behavioral, and signature-based methods. We further explore the potential of complementary mechanisms such as watermarking and trusted execution environments to enhance verifiability without compromising provider confidentiality. We also propose a modular three-layer auditing framework for COLS and users that enables trustworthy verification across execution, secure logging, and user-facing auditability without exposing proprietary internals. Our aim is to encourage further research and policy development toward transparency, auditability, and accountability in commercial LLM services.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。