发现大模型推理时概率变换有稳定对数关系，可量化分析推理过程。
Empirical Characterization of Inference-Time Elicited Probability Transformations in Large Language Models
- 通过外部证据信号建模推理中概率变化规律，发现近似对数线性关系。
- 在1.3万次候选答案评估中，关系拟合度平均达R²≈0.76，跨任务和模型一致。
- 适用于研究模型校准、证据增强与不确定性传播，适合方法设计者参考。
大型语言模型越来越多地依赖推理时方法，如思维链、自精炼、检索增强和验证器引导修订，但这些方法引发的概率变换结构仍不清晰。本文研究了外部诱导的候选答案概率分配，观察到重复出现的近似对数比率关系： log q̃_t(i) = α_t (log q_t(i) + log b_t(i)) + c_t 其中，q_t 和 q̃_t 分别为诱导前后的概率，b_t 是外部构建的证据信号，α_t 为提示配置的经验描述符。我们在来自 GPQA Diamond、TheoremQA、MMLU-Pro 与 ARC-Challenge 的 4,975 个推理问题上，对多个指令微调模型族进行评估，共获得约 1.3×10⁵ 个候选级观测，发现该关系平均拟合度 R² ≈ 0.76。系数随诱导设置变化，但定性相似关系在各类条件下持续存在。通过替代统计表示、不同提示配置、留出测试及词元级对数概率的鲁棒性分析表明，该结构并非特定于某一种提示或概率估计方式。核心贡献不在于代数形式本身（其与广义贝叶斯更新相关），而在于实证发现：多样化的推理时提示流程在受控条件下反复表现出可复现的对数比率结构。该框架为分析校准、证据放大、不确定性传播与交互敏感性提供了协议感知视角。
原文摘要 · Abstract (English)
Large language models increasingly rely on inference-time procedures such as chain-of-thought reasoning, self-refinement, retrieval augmentation, and verifier-guided revision, yet the structure of elicited probability transformations under these procedures remains poorly understood. We study externally elicited probability assignments over candidate answers and observe recurring approximate log-ratio relationships: \[ \log \tilde q_t(i) = α_t \left( \log q_t(i) + \log b_t(i) \right) + c_t, \] where $q_t$ and $\tilde q_t$ are pre- and post-elicitation probabilities, $b_t$ is an externally constructed evidence signal, and $α_t$ is an empirical descriptor of the prompting configuration. Across 4,975 reasoning problems from GPQA Diamond, TheoremQA, MMLU-Pro, and ARC-Challenge, evaluated on multiple instruction-tuned model families, we observe approximate log-ratio relationships with mean $R^2 \approx 0.76$ over about $1.3 \times 10^5$ candidate-level observations. Coefficients vary across elicitation settings, but qualitatively similar relationships persist across evaluated conditions. Robustness analyses using alternative statistical representations, prompting configurations, held-out evaluation, and token-level log-probabilities suggest that the observed structure is not tied to one prompting procedure or probability estimation method. The main contribution is not the algebraic form itself, which is related to generalized Bayesian updating and probability-transformation frameworks, but the empirical observation that diverse inference-time prompting pipelines repeatedly exhibit reproducible log-ratio structure under controlled conditions. The framework provides a protocol-sensitive perspective for analyzing calibration, evidence amplification, uncertainty propagation, and interaction sensitivity in inference-time LLM pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。