arXiv:2608.17638cs.AI2026-08

让专家模型的推理过程可读可操作,提升决策准确率

Beyond the Trace: Coupling an Interpretable Reasoning-State Readout to Native MoE Routing

  • 用语义空间J64解析模型内部推理状态,揭示输出文本未体现的思维过程
  • 相比基线,预测性能提升0.096至0.135的AUC,单分支选择准确率提高1.1至5.9点
  • 通过原生路由统计重建的R64代理低开销、高保真,适用于实时决策

模型输出仅是推理过程的部分记录。本文提出两级内部读出机制:首先将词汇级语义空间降维为由模型自身推理状态学习得到的64维语义框架J64,其可读性强,能区分推理努力与问题压力;在相同聚合方式下,相较于仅读取词元占用的基线,其在保留测试集上的AUC提升0.096至0.135。其次,从原生专家路由统计中重构出低开销代理R64,其各轴与J64的中位相关性达0.69至0.86,在gpt-oss-20b上保留了J64 95%至100%的预测增益。该读出支持双时间尺度的测试时决策:在候选集完成后,J64和R64均提升单分支选择效果,且R64加权投票在八组设置中七组优于普通多数投票;生成过程中,滚动窗口驱动累积停止并重采样策略,仅在训练问题上校准,使准确率提升1.1至5.9点,而仅依赖路由的R64仍保留0.9至3.2点增益。最后,针对J64所标识机制进行路由修改,可诱发预期推理行为,并将诊断出的卡顿从数值猜测转为精确符号执行。总体而言,J64使隐含推理状态可读,而路由使其可部署、可行动。

原文摘要 · Abstract (English)

What a reasoning model writes is only a partial record of the process that produces it. We introduce a two-level internal readout for mixture-of-experts reasoning. We first distill vocabulary-scale J-space into J64, a 64-axis semantic frame learned from the model's own reasoning states. J64 reveals readable process state that the emitted trace does not show: it separates inference effort from problem-induced strain. It also adds 0.096 to 0.135 held-out AUC over a baseline that reads the same rollout as token occupancy and aggregates it in exactly the same way. We then reconstruct J64 from native expert-routing statistics. The result is R64, a low-overhead proxy: its median per-axis correlation with J64 is 0.69 to 0.86 across three models and two families, and on gpt-oss-20b it preserves 95 to 100% of J64's predictive gain. The readout supports test-time decisions at two temporal resolutions. Over completed candidate sets, J64 and R64 improve single-branch selection, and R64-weighted voting improves plain majority voting in seven of eight settings. During generation, rolling readout windows drive a cumulative stop-and-resample policy whose operating point is fixed on training questions alone. J64 improves accuracy by 1.1 to 5.9 points over a sibling-permuted control, and the routing-only R64 proxy retains 0.9 to 3.2 of those points. Finally, router edits aimed at the mechanism J64 names induce the predicted reasoning behaviors and shift a diagnosed stall from numerical guessing toward exact symbolic execution. Together, J64 makes latent process state readable, while routing makes it deployable and actionable.

MoE可解释性推理增强路由优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。