用户可预测大模型隐藏推理耗的token数,防被超收
Predictive Auditing of Hidden Tokens in LLM APIs via Reasoning Length Estimation
- 用轻量路由模块动态校准不同任务的推理耗token
- 在数学、编程等4个任务上误差低,能精准估算隐藏消耗
- 适合关心算力账单透明度的开发者与企业用户
商用大模型服务常隐藏内部推理过程,却仍对所有生成的token(包括中间步骤)计费,引发令牌膨胀与潜在超收问题。现有审计手段受限:密码验证因服务商控制全程而无效,用户侧预测又受大模型推理波动影响。为此,我们提出PALACE(Predictive Auditing of LLM APIs via Reasoning Token Count Estimation),一种无需访问内部日志即可从提示-回答对中估算隐藏推理token数的用户端框架。PALACE引入基于GRPO的适配模块与轻量级领域路由机制,实现跨多种推理任务的动态校准,有效缓解token使用模式的波动性。在数学、编程、医疗及通用推理基准测试中,PALACE均表现出低相对误差和高预测精度,支持细粒度成本审计与通胀检测。本工作为标准化预测审计迈出关键一步,助力提升透明度、问责性与用户信任。
原文摘要 · Abstract (English)
Commercial LLM services often conceal internal reasoning traces while still charging users for every generated token, including those from hidden intermediate steps, raising concerns of token inflation and potential overbilling. This gap underscores the urgent need for reliable token auditing, yet achieving it is far from straightforward: cryptographic verification (e.g., hash-based signature) offers little assurance when providers control the entire execution pipeline, while user-side prediction struggles with the inherent variance of reasoning LLMs, where token usage fluctuates across domains and prompt styles. To bridge this gap, we present PALACE (Predictive Auditing of LLM APIs via Reasoning Token Count Estimation), a user-side framework that estimates hidden reasoning token counts from prompt-answer pairs without access to internal traces. PALACE introduces a GRPO-augmented adaptation module with a lightweight domain router, enabling dynamic calibration across diverse reasoning tasks and mitigating variance in token usage patterns. Experiments on math, coding, medical, and general reasoning benchmarks show that PALACE achieves low relative error and strong prediction accuracy, supporting both fine-grained cost auditing and inflation detection. Taken together, PALACE represents an important first step toward standardized predictive auditing, offering a practical path to greater transparency, accountability, and user trust.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。