用简单提示词让大模型生成可验证的推理链条，5分靠前。
LLMSR@XLLM25: An Empirical Study of LLM for Structural Reasoning
- 设计多轮提示词，引导模型逐步标注证据与结论
- 无需微调即达5名，宏平均F1媲美复杂系统
- 适合关注可解释推理的AI研究者
我们提交团队asdfo123的方案参加LLMSR@XLLM25共享任务，评估大语言模型生成细粒度、可控且可解释推理过程的能力。系统需提取所有问题条件，将思维链分解为陈述-证据对，并验证每对逻辑有效性。仅使用预训练的Meta-Llama-3-8B-Instruct模型，通过简洁的少样本多轮提示，先枚举全部条件，再引导模型标注、引用并判断每一步推理。基于正则表达式的轻量后处理模块统一文本跨度并强制符合官方JSON格式。无需微调、外部检索或集成，方法在整体排名中位列第5，宏平均F1得分与更复杂、资源消耗更大的流水线相当。最后分析了该方法的优势与局限，并提出未来结构化推理研究方向。代码已开源：https://github.com/asdfo123/LLMSR-asdfo123。
原文摘要 · Abstract (English)
We present Team asdfo123's submission to the LLMSR@XLLM25 shared task, which evaluates large language models on producing fine-grained, controllable, and interpretable reasoning processes. Systems must extract all problem conditions, decompose a chain of thought into statement-evidence pairs, and verify the logical validity of each pair. Leveraging only the off-the-shelf Meta-Llama-3-8B-Instruct, we craft a concise few-shot, multi-turn prompt that first enumerates all conditions and then guides the model to label, cite, and adjudicate every reasoning step. A lightweight post-processor based on regular expressions normalises spans and enforces the official JSON schema. Without fine-tuning, external retrieval, or ensembling, our method ranks 5th overall, achieving macro F1 scores on par with substantially more complex and resource-consuming pipelines. We conclude by analysing the strengths and limitations of our approach and outlining directions for future research in structural reasoning with LLMs. Our code is available at https://github.com/asdfo123/LLMSR-asdfo123.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。