用双系统框架让问答模型既懂语言又会逻辑推理。
Speaking in Words, Thinking in Logic: A Dual-Process Framework in QA Systems
- 借鉴认知双系统理论,分步实现自然语言到逻辑的转换。
- 在教育医疗等领域达成与大模型相当的推理准确率,耗时更短。
- 适合需要透明解释的高风险领域,如法律、医疗问答。
大型语言模型(LLMs)显著提升了开放域问答能力,但在教育、医疗、法律等封闭域中,用户不仅要求答案准确,还期待推理过程透明可解释。尽管神经符号(NeSy)框架结合了语言理解与形式化推理,但现有方法多依赖大规模模型,且自然语言转逻辑表示效率低。为此,我们提出Text-JEPA:一种轻量级文本联合嵌入预测架构,可将自然语言转化为一阶逻辑(NL2FOL)。该框架受双系统认知理论启发,由高效生成逻辑表示的System 1与执行稳健逻辑推理的Z3求解器(System 2)协同工作。为全面评估从NL2FOL到推理的全流程性能,我们设计三类自定义指标:转换得分、推理得分与斯皮尔曼等级相关系数得分,综合衡量逻辑转化质量及其对推理准确率的影响。在多个领域特定数据集上的实验证明,Text-JEPA在保持竞争力的同时,计算开销远低于基于大模型的系统。研究结果表明,结构化、可解释的推理框架在专业领域中具有构建高效可信问答系统的重要潜力。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have significantly enhanced question-answering (QA) capabilities, particularly in open-domain contexts. However, in closed-domain scenarios such as education, healthcare, and law, users demand not only accurate answers but also transparent reasoning and explainable decision-making processes. While neural-symbolic (NeSy) frameworks have emerged as a promising solution, leveraging LLMs for natural language understanding and symbolic systems for formal reasoning, existing approaches often rely on large-scale models and exhibit inefficiencies in translating natural language into formal logic representations. To address these limitations, we introduce Text-JEPA (Text-based Joint-Embedding Predictive Architecture), a lightweight yet effective framework for converting natural language into first-order logic (NL2FOL). Drawing inspiration from dual-system cognitive theory, Text-JEPA emulates System 1 by efficiently generating logic representations, while the Z3 solver operates as System 2, enabling robust logical inference. To rigorously evaluate the NL2FOL-to-reasoning pipeline, we propose a comprehensive evaluation framework comprising three custom metrics: conversion score, reasoning score, and Spearman rho score, which collectively capture the quality of logical translation and its downstream impact on reasoning accuracy. Empirical results on domain-specific datasets demonstrate that Text-JEPA achieves competitive performance with significantly lower computational overhead compared to larger LLM-based systems. Our findings highlight the potential of structured, interpretable reasoning frameworks for building efficient and explainable QA systems in specialized domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。