LLM在推理中依赖中间结构，但修改后却不会改变结果，揭示其表面一致性实则脆弱。
Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structures
- 通过因果干预测试，检验模型对中间结构的响应能力
- 12个模型在4个基准上均未因结构修改而调整最终决策
- 外部工具可显著提升干预一致性，适合可控推理场景
在基于模式的推理（SGR）流程中，大语言模型会生成显式的中间结构，如检查清单或验证查询，再做出最终判断。这类方法被认为能提升可控性，允许用户检查、修改或覆盖中间结果以引导输出。但这种承诺是否成立？我们提出一种因果评估协议：在特定任务中，中间结构与最终决策间存在确定性映射，每次可控修改都应对应唯一正确输出。在12个模型和4个基准上的实验表明，模型看似与其自身中间结构保持一致，但在干预后并未更新预测结果——说明其表面一致性在结构变化后极为脆弱。当最终决策由外部工具从中间结构推导时，这种脆弱性几乎消失；强化提示仅带来有限改进，而偏好优化则显著提升干预后的忠实度。总体而言，中间结构在推理中更像有影响力的上下文，而非稳定的因果中介。
原文摘要 · Abstract (English)
In schema-guided reasoning (SGR) pipelines, LLMs produce explicit intermediate structures -- rubrics, checklists, or verification queries -- before committing to a final decision. SGR is increasingly adopted because it promises controllability: practitioners expect to inspect, edit, and override these structures to steer the outcome. But does the promise hold? We introduce a causal evaluation protocol to measure it: by selecting tasks where a deterministic function maps intermediate structures to decisions, every controlled edit implies a unique correct output. Across 12 models and 4 benchmarks, models appear self-consistent with their own intermediate structures but fail to update predictions after intervention -- revealing that apparent faithfulness is fragile once the intermediate structure changes. When derivation of the final decision from the structure is delegated to an external tool, this fragility largely disappears; stronger prompting yields only limited improvements, while preference optimization substantially improves intervention faithfulness. Overall, intermediate structures in schema-guided pipelines function as influential context rather than stable causal mediators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。