用代数方法增强大模型推理链,让复杂问题分解更系统、更准确。
Syzygy of Thoughts: Improving LLM CoT with the Minimal Free Resolution
- 引入代数中的极小自由分解思想,构建多条关联推理路径。
- 在GSM8K等数据集上表现优于主流推理方法,准确率更高。
- 适合需要高可靠推理的科研与工程场景,尤其擅长复杂逻辑题。
思维链(CoT)提示通过将问题分解为一系列步骤来提升大语言模型的推理能力,模拟人类逻辑并减少错误。然而,面对解空间庞大且约束模糊的复杂任务时,单一推理链往往力不从心。受交换代数与代数几何中极小自由分解(MFR)的启发,我们提出「思维的结合」(Syzygy of Thoughts, SoT)——一种扩展CoT的新框架,引入相互关联的辅助推理路径,以捕捉更深层的逻辑依赖关系,实现更鲁棒、结构化的求解。MFR将模分解为秩最小的自由模序列,提供分析复杂系统的系统性方法。该方法引入了“模”、“Betti数”、“自由性”、“映射”、“精确性”和“最小性”等概念,将原始复杂问题系统性地分解为逻辑完备的最小子问题,同时保留关键特征并缩短推理长度。我们在多个数据集(如GSM8K、MATH)和模型(如GPT-4o-mini、Qwen2.5)上验证了SoT,其推理准确率达到或超越主流CoT基准。此外,通过将采样过程与代数约束对齐,我们的方法提升了推理时间的可扩展性,兼顾推理透明性与高性能。代码将公开于 https://github.com/dlMARiA/Syzygy-of-thoughts。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) prompting enhances the reasoning of large language models (LLMs) by decomposing problems into sequential steps, mimicking human logic and reducing errors. However, complex tasks with vast solution spaces and vague constraints often exceed the capacity of a single reasoning chain. Inspired by Minimal Free Resolution (MFR) in commutative algebra and algebraic geometry, we propose Syzygy of Thoughts (SoT)-a novel framework that extends CoT by introducing auxiliary, interrelated reasoning paths. SoT captures deeper logical dependencies, enabling more robust and structured problem-solving. MFR decomposes a module into a sequence of free modules with minimal rank, providing a structured analytical approach to complex systems. This method introduces the concepts of "Module", "Betti numbers","Freeness", "Mapping", "Exactness" and "Minimality", enabling the systematic decomposition of the original complex problem into logically complete minimal subproblems while preserving key problem features and reducing reasoning length. We tested SoT across diverse datasets (e.g., GSM8K, MATH) and models (e.g., GPT-4o-mini, Qwen2.5), achieving inference accuracy that matches or surpasses mainstream CoTs standards. Additionally, by aligning the sampling process with algebraic constraints, our approach enhances the scalability of inference time in LLMs, ensuring both transparent reasoning and high performance. Our code will be publicly available at https://github.com/dlMARiA/Syzygy-of-thoughts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。