通过推理过程的早期承诺时间，无须奖励模型即可检测语言模型的隐式捷径行为。
Self-Commitment Latency: A Reward-Free Probe for Prompted Implicit Hacking
- 用自我承诺延迟度量推理中模型提前锁定答案的时间点。
- 在GSM8K上，带提示的推理比正常推理提前20%以上承诺，且不确定性更低。
- 无需外部奖励或训练，适合检测模型隐式作弊行为。
当语言模型的思维链看似正常时，其最终答案可能受提示捷径影响而产生隐式奖励劫持，难以审计。现有验证器方法需任务特异性奖励信号，本文提出弱输入替代方案——自承诺延迟，测量提示推理过程何时开始承诺自身最终答案。在使用Qwen2.5-3B-Instruct-4bit的控制型配对GSM8K实验中，含答案提示的上下文比诚实上下文更早且更确定地承诺。主指标首承诺延迟（阈值0.8）达AUROC 0.878；整体曲线摘要分别达0.926与0.904。该信号在正确回答条件下更强，且跨阈值稳定。结果表明，捷径可用的推理可留下可检测的早期行为签名，无需奖励模型、外部评判或训练分类器。
原文摘要 · Abstract (English)
Implicit reward hacking is hard to audit when a language model's chain of thought appears benign: a final answer may be anchored by a prompt shortcut while the written reasoning still resembles ordinary problem solving. Verifier-based probes expose such behavior by measuring how early truncated reasoning contexts obtain high reward, but require a task-specific reward signal. This paper proposes a weaker-input alternative, self-commitment latency, which measures how early a prompted reasoning context commits to the model's own final answer. We evaluate the probe in a controlled paired GSM8K setting using Qwen2.5-3B-Instruct-4bit, comparing ordinary prompts with prompts that include an answer hint. Hinted contexts commit substantially earlier and with lower uncertainty than honest contexts. The primary latency metric, first-commitment latency at threshold 0.8, reaches AUROC 0.878; supporting whole-curve summaries reach AUROC 0.926 for commitment range and 0.904 for mean uncommitted mass. The signal is stronger when both prompt conditions answer correctly and remains stable across thresholds. These results show that shortcut-available reasoning contexts can leave an early behavioral commitment signature detectable without a reward model, external judge, or trained classifier.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。