发现大模型多阶段流程中纠错失效的核心机制
Detection Without Correction: A Two-Parameter Decomposition of Multi-Stage LLM Pipelines
- 将下游响应拆解为检测与生成两个耦合决策
- 94%的错误来自误判后仍继续生成，而非检测失败
- 该现象在四类模型、四大评测中普遍出现，适合研究推理机制者关注
多阶段大模型流水线（如多智能体辩论、内在自纠正、检索增强验证）表现出令人困惑的聚合行为：准确率在轮次间平台化甚至反转，辩论增益在前沿模型上无法复现，自纠正能力下降，且不同厂商的辩论动态差异显著。下游响应可被建模为两个耦合决策：检测（是否将上游内容视为权威）与条件生成（若不认可则生成什么）。此分解揭示四种可观测响应模式，其中‘仅检测不纠正’是核心失效模式。在覆盖四类模型、四个基准（GSM8K, MATH-500, GPQA-Diamond, AIME）及两种方法（多智能体辩论、内在自纠正）的九格实验中，条件性误纠正率始终占主导（各群体53%-94%），而检测率则因情境差异超过一个数量级。该框架将前述四种现象统一为同一机制的表征，并指出检测阈值是跨方法、同难度下稳定的模型/协议级规律。
原文摘要 · Abstract (English)
Multi-stage LLM pipelines that perform multi-agent debate, intrinsic self-correction, or retrieval-augmented verification exhibit puzzling aggregate behaviors: accuracy plateaus and reversals across rounds, non-replication of debate gains on contemporary frontier models, intrinsic self-correction degradation, and qualitative cross-provider divergence in debate dynamics. Downstream agent response can be operationalized as two coupled decisions: detection (whether to treat upstream content as authoritative) and conditional generation (what to produce if not). This decomposition yields four observable response regimes, of which detection-without-correction is the load-bearing failure mode. Across a nine-cell empirical grid spanning four model families, four benchmarks (GSM8K, MATH-500, GPQA-Diamond, AIME), and two methods (multi-agent debate, intrinsic self-correction), we find that the conditional miscorrection rate is consistently dominant (53-94% across cohorts) while detection rate varies contextually by more than an order of magnitude. The framework unifies the four phenomena above as signatures of a common mechanism and characterizes detection threshold as a stable model/protocol-level regularity that persists across methods at matched benchmark difficulty.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。