arXiv:2507.03347cs.AI2025-07

无结构推理在复杂任务中表现更优,挑战了结构化输出的必要性。

Effects of structure on reasoning in instance-level Self-Discover

  • 对比动态生成的结构化与非结构化推理路径
  • 无结构推理在MATH上提升达18.90%
  • 适合追求高效推理的系统设计者参考

为实现大模型在复合系统中可预测的推理,结构化输出广受青睐,但其性能常逊于自由文本。同时,基于自由链式思维(CoT)训练的模型虽强,却带来计算开销和忠实度挑战。本文提出iSelf-Discover,一种实例级自发现框架,对比动态生成的结构化JSON推理与非结构化推理。在多个基准测试中,使用开源先进模型的实证评估显示,非结构化推理持续占优。尤其在复杂MATH基准上,非结构化方案相对结构化方案最高提升18.90%。零样本非结构化变体甚至优于五样本结构化版本,表明即使推理先于答案生成,该差距依然显著。此外,计划粒度(实例级与任务级)的最优选择取决于具体上下文。这些发现呼吁重新审视复杂问题求解中对结构化格式的依赖,以及复合系统的组织方式。

原文摘要 · Abstract (English)

The drive for predictable LLM reasoning in their integration with compound systems has popularized structured outputs, yet concerns remain about performance trade-offs compared to unconstrained natural language. At the same time, training on unconstrained Chain of Thought (CoT) traces has brought about a new class of strong reasoning models that nevertheless present novel compute budget and faithfulness challenges. This paper introduces iSelf-Discover, an instance-level adaptation of the Self-Discover framework, and using it compares dynamically generated structured JSON reasoning with its unstructured counterpart. Our empirical evaluation across diverse benchmarks using state-of-the-art open-source models supports a consistent advantage for unstructured reasoning. Notably, on the complex MATH benchmark, unstructured plans achieved relative performance improvements of up to 18.90\% over structured approaches. Zero-shot unstructured iSelf-Discover variants are also shown to outperform their five-shot structured counterparts, underscoring the significance of this gap, even when structured plans are dynamically generated to ensure reasoning precedes the final answer. We further demonstrate that the optimal granularity of plan generation (instance-level vs. task-level) is context-dependent. These findings invite re-evaluation of the reliance on structured formats for complex problem-solving and how compound systems should be organized.

推理机制结构化输出大模型MATH基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。