用概率推理让大模型更准地处理带逻辑条件的复杂查询。
OrLog: Resolving Complex Queries with LLMs and Probabilistic Reasoning
- 先用大模型快速评估每个条件真假,再用概率方法组合结果。
- 在包含或关系的查询上,准确率提升明显,且每查询少用90% token。
- 适合需要精确逻辑判断的信息检索任务,如智能搜索、问答系统。
复杂信息需求常涉及多个约束条件,需在候选答案集上正确施加逻辑运算(合取、析取、否定)。当前检索系统要么忽略神经嵌入中的逻辑约束,要么在生成式推理中近似处理,导致不一致与不可靠。虽神经符号方法适用于结构化推理,但多局限于形式逻辑或数学问题,依赖明确查询与完整证据,这在实际信息检索中罕见。为此,我们提出OrLog:一种将谓词可信度估计与逻辑推理解耦的神经符号检索框架。大语言模型(LLM)在一次无生成前向传播中输出原子谓词的可信度分数,再由概率推理引擎推导查询满足的后验概率。我们在多种主干LLM、不同外部知识访问程度及多样逻辑约束下评估OrLog,对比基础检索器与LLM作为推理机的方法。当提供实体描述时,OrLog相比传统LLM推理显著提升高排名精度,尤其在析取查询上增益更大;同时,平均每个查询-实体对减少约90%的token使用。结果表明,无需生成的谓词可信度估计结合概率推理,能实现更高效、更准确的约束感知检索,优于整体式推理,且资源消耗极低。
原文摘要 · Abstract (English)
Resolving complex information needs that come with multiple constraints should consider enforcing the logical operators encoded in the query (i.e., conjunction, disjunction, negation) on the candidate answer set. Current retrieval systems either ignore these constraints in neural embeddings or approximate them in a generative reasoning process that can be inconsistent and unreliable. Although well-suited to structured reasoning, existing neuro-symbolic approaches remain confined to formal logic or mathematics problems as they often assume unambiguous queries and access to complete evidence, conditions rarely met in information retrieval. To bridge this gap, we introduce OrLog, a neuro-symbolic retrieval framework that decouples predicate-level plausibility estimation from logical reasoning: a large language model (LLM) provides plausibility scores for atomic predicates in one decoding-free forward pass, from which a probabilistic reasoning engine derives the posterior probability of query satisfaction. We evaluate OrLog across multiple backbone LLMs, varying levels of access to external knowledge, and a range of logical constraints, and compare it against base retrievers and LLM-as-reasoner methods. Provided with entity descriptions, OrLog can significantly boost top-rank precision compared to LLM reasoning with larger gains on disjunctive queries. OrLog is also more efficient, cutting mean tokens by $\sim$90\% per query-entity pair. These results demonstrate that generation-free predicate plausibility estimation combined with probabilistic reasoning enables constraint-aware retrieval that outperforms monolithic reasoning while using far fewer tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。