用时序逻辑量化思维链每步可信度，提升大模型推理可靠性
Temporalizing Confidence: Evaluation of Chain-of-Thought Reasoning with Signal Temporal Logic
- 将思维链每一步的置信度建模为时序信号，用时序逻辑约束其动态特性
- 在数学推理任务中，置信度校准误差降低23.7%，优于传统方法
- 适合教育、医疗等需可解释推理的高风险场景
大型语言模型在链式思维提示引导下，在数学推理任务中表现优异。然而，它们常生成高度自信却错误的输出，这在教育等用户缺乏评估能力的领域带来显著风险。为此，我们提出一种结构化框架，将逐步置信度建模为时序信号，并利用信号时序逻辑（STL）进行评估。具体而言，我们定义了基于STL的正式约束以捕捉理想的时序性质，并计算鲁棒性得分作为结构化、可解释的置信度估计。该方法还引入一系列不确定性重塑策略，以确保推理轨迹中的平滑性、单调性和因果一致性。实验表明，该方法在多个校准指标上持续优于传统的置信度聚合与事后校准方法，提供了更可靠的不确定性估计。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown impressive performance in mathematical reasoning tasks when guided by Chain-of-Thought (CoT) prompting. However, they tend to produce highly confident yet incorrect outputs, which poses significant risks in domains like education, where users may lack the expertise to assess reasoning steps. To address this, we propose a structured framework that models stepwise confidence as a temporal signal and evaluates it using Signal Temporal Logic (STL). In particular, we define formal STL-based constraints to capture desirable temporal properties and compute robustness scores that serve as structured, interpretable confidence estimates. Our approach also introduces a set of uncertainty reshaping strategies to enforce smoothness, monotonicity, and causal consistency across the reasoning trajectory. Experiments show that our approach consistently improves calibration metrics and provides more reliable uncertainty estimates than conventional confidence aggregation and post-hoc calibration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。