强制结构化输出反而让大模型陷入格式陷阱,导致错误越纠越多。
From Hallucination to Structure Snowballing: The Alignment Tax of Constrained Decoding in LLM Reflection
- 用大纲约束解码强迫模型按格式生成
- 模型表面语法正确,但深层错误未被发现
- 适合研究自主智能体与纠错机制的学者
大语言模型在开放推理任务中常因「幻觉雪球效应」而自我修正失败,即模型在自由文本反思中反复为早期错误辩护。尽管结构化反馈可缓解此问题,现有方法多依赖外部训练的评判器或符号工具,削弱了智能体自主性。本研究探讨仅通过基于大纲的约束解码是否能在不额外训练的情况下阻断错误传播。以80亿参数模型Qwen3-8B为实验对象,结果表明单纯施加结构约束不仅未能提升自纠正能力,反而引发新的失效模式——「结构雪球效应」。我们发现,严格格式要求带来的认知负荷使模型陷入格式陷阱。这一现象解释了为何智能体在表面语法对齐上近乎完美,却无法察觉或修复深层语义错误。研究揭示了约束解码内在的「对齐税」:结构粒度与模型内部容量之间存在张力,影响自主工作流的可靠性。代码与原始日志见GitHub仓库:https://github.com/hongxuzhou/agentic_llm_structured_self_critique。
原文摘要 · Abstract (English)
Intrinsic self-correction in Large Language Models (LLMs) frequently fails in open-ended reasoning tasks due to ``hallucination snowballing,'' a phenomenon in which models recursively justify early errors during free-text reflection. While structured feedback can mitigate this issue, existing approaches often rely on externally trained critics or symbolic tools, reducing agent autonomy. This study investigates whether enforcing structured reflection purely through Outlines-based constrained decoding can disrupt error propagation without additional training. Evaluating an 8-billion-parameter model (Qwen3-8B), we show that simply imposing structural constraints does not improve self-correction performance. Instead, it triggers a new failure mode termed ``structure snowballing.'' We find that the cognitive load required to satisfy strict formatting rules pushes the model into formatting traps. This observation helps explain why the agent achieves near-perfect superficial syntactic alignment yet fails to detect or resolve deeper semantic errors. These findings expose an ``alignment tax'' inherent to constrained decoding, highlighting a tension between structural granularity and internal model capacity in autonomous workflows. Code and raw logs are available in the GitHub repository: https://github.com/hongxuzhou/agentic_llm_structured_self_critique.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。