用LLM从自然语言生成逻辑图，提升机器推理能力。
Benchmarking graph construction by large language models for coherence-driven inference
- 设计算法生成可支撑连贯推理的命题，构建逻辑图
- o1/3/4-mini在稀疏图上单次提示下一半情况完美重建
- 适合研究大模型推理与认知能力的学者关注
我们设计了一种算法，用于生成客观实例化的命题，以构建支持连贯推理的图结构。同时，我们评估了大语言模型（LLMs）从自然语言命题（经简单转换后）重建连贯性图的能力，结果表明，仅通过一次提示，优化推理的LLM便能取得良好表现。例如，o1/3/4-mini在稀疏图上半数情况下实现完美重建。基于连贯性评估的连贯推理可能推动机器认知能力的发展。
原文摘要 · Abstract (English)
We devise an algorithm to generate propositions that objectively instantiate graphs supporting coherence-driven inference. We also benchmark the ability of large language models (LLMs) to reconstruct coherence graphs from (a simple transformation of) propositions expressed in natural language, with promising results from a single prompt to reasoning-optimized LLMs. For example, o1/3/4-mini achieve perfect reconstruction half of the time on sparse graphs. Coherence-driven inference on consistency evaluations by LLMs may advance machine cognition capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。