用结构化提示让大模型更懂规则，推理更透明可信。
Explainable Rule Application via Structured Prompting: A Neural-Symbolic Approach
- 将推理拆解为实体识别、属性提取和符号规则应用三步，确保逻辑可验证。
- 在法律传闻证据任务中，o1模型F1达0.929，显著优于基线的0.714。
- 适合需要可解释性与严格规则执行的法律、医疗等专业领域使用。
大型语言模型在复杂推理任务中表现优异，但在规则一致应用、例外处理和可解释性方面存在不足，尤其在需要自然语言理解与精确逻辑推理结合的法律分析领域。本文提出一种结构化提示框架，将推理分解为三个可验证步骤：实体识别、属性提取和符号规则应用。通过融合神经与符号方法，该框架利用大模型的解释灵活性，同时借助形式化验证保障逻辑一致性。任务定义被外部化，使领域专家可在不修改架构的情况下优化逻辑结构。在LegalBench传闻证据判定任务上的评估显示，采用结构化分解与互补谓词的OpenAI o-family模型性能显著提升：o1达到F1 0.929，o3-mini达0.867，远超其少样本基线（0.714 和 0.74）。该混合神经-符号系统为透明且一致的规则推理提供了可行路径，适用于需要可解释AI的结构化法律推理场景。
原文摘要 · Abstract (English)
Large Language Models (LLMs) excel in complex reasoning tasks but struggle with consistent rule application, exception handling, and explainability, particularly in domains like legal analysis that require both natural language understanding and precise logical inference. This paper introduces a structured prompting framework that decomposes reasoning into three verifiable steps: entity identification, property extraction, and symbolic rule application. By integrating neural and symbolic approaches, our method leverages LLMs' interpretive flexibility while ensuring logical consistency through formal verification. The framework externalizes task definitions, enabling domain experts to refine logical structures without altering the architecture. Evaluated on the LegalBench hearsay determination task, our approach significantly outperformed baselines, with OpenAI o-family models showing substantial improvements - o1 achieving an F1 score of 0.929 and o3-mini reaching 0.867 using structured decomposition with complementary predicates, compared to their few-shot baselines of 0.714 and 0.74 respectively. This hybrid neural-symbolic system offers a promising pathway for transparent and consistent rule-based reasoning, suggesting potential for explainable AI applications in structured legal reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。