检测手机端语言模型在无监督情况下的隐藏错误，发现其自信出错无法识别。
Confidently Wrong, Silently So: Auditing Undetectable Failures of a Deployed On-Device Language Model
- 通过红队测试验证模型在虚假前提下会自信编造答案
- 模型对无害提示拒绝率高达18%，而自信正确与错误输出外表相同
- 提出无需模型访问的黑箱一致性检测法，可显著提升可靠性
部署的语言模型需判断输出可信度，但如今数亿设备上的本地模型缺乏服务器端审核，开发者实际部署配置极少被独立审计。本文对可访问的本地基础模型开展可复现的可靠性审计，核心问题是：用户或资源受限开发者能否察觉模型出错？通过校准性、虚假前提下的自信编造、对无害提示的过度拒绝等红队测试，发现存在任务不对称校准偏差：在69%的虚假前提下自信编造，同时拒绝18%的完全无害输入；模型自报置信度饱和且无区分能力（AUROC 0.47；ECE 70，为同类小型模型中最差）。关键发现：自信正确与自信错误输出表面不可区分——基于15个用户可见特征的分类器仅达AUROC 0.55（经等效性验证），推理时无监督信号可用。单一生成信号效果差（≤0.68 AUROC），而无需模型访问的黑箱一致性封装可恢复可靠性（自信编造从69%降至3%；选择性准确率从43%升至83%），成本可调。贡献包括模型无关审计协议、表面不可区分性测试，以及可复用的代码和冻结评估项。
原文摘要 · Abstract (English)
Aligning deployed language models requires knowing when their outputs can be trusted, yet on-device models now ship to hundreds of millions of devices with no server-side moderation, and the configuration developers can actually deploy is rarely audited independently. We present a reproducible reliability audit of the developer-accessible on-device foundation model, framed as an oversight question: can a user or a resource-constrained developer tell when the model is wrong? Red-teaming it on calibration, confident confabulation on false-premise questions, and over-refusal of benign prompts, we find a \emph{task-asymmetric miscalibration}: its guardrails fail in opposite directions across tasks (confabulating on 69\% of false premises while refusing 18\% of entirely benign inputs), atop a self-reported confidence that is saturated and non-discriminative (AUROC 0.47; ECE 70, worst among comparable small models). Crucially, confident-correct and confident-wrong outputs are \emph{surface-indistinguishable}: a classifier over 15 user-visible features separates them at AUROC only 0.55 (equivalence-confirmed), leaving no signal for oversight at inference time. No cheap single-generation signal flags these failures ($\le$0.68 AUROC), whereas a black-box consistency wrapper requiring no model access recovers reliability (confident confabulation 75\%$\to$3\%; selective accuracy 43\%$\to$83\%) at a tunable cost. We contribute a model-agnostic audit protocol, a surface-indistinguishability test, and released code and frozen evaluation items as reusable infrastructure for auditing deployed models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。