让大模型在法律推理中更准地选证据,避免被多数错误答案带偏。
Quantum-Inspired Trace-Augmented Evidence Selection for Reasoning over Structured Hypothesis Spaces
- 把推理片段选证据变成优化问题,用质量权重筛选最优组合。
- 在两个法律数据集上,相比投票法提升6.2%准确率,低污染场景效果更显著。
- 适合需要精细证据判断的领域,如法律、医学等专业推理任务。
大型语言模型已在诸多专家级考试中达到或超过人类水平,但在法律等高度依赖证据的专业领域仍表现脆弱。错误不仅源于知识缺口,也来自证据间细微差异和使用不一致。当前主流的链式思维(CoT)采样聚合方式——多数投票——会忽略真正有力但少数的正确证据。本文提出将推理片段选择转化为显式的组合优化问题,使高质量但少数的假设可胜过嘈杂多数。我们引入EP-HUBO(证据池高阶二元优化),通过小型局部模型生成多个CoT路径,将碎片解析为每假设对应的证据池,基于相关性、特异性、独特性等质量权重,对每个证据池求解高阶无约束二元优化,并仅调用前沿模型进行一次判决。在两个对证据质量极为敏感的法律推理基准上评估,分别使用经典硬件模拟退火与量子计算公司推出的Dirac-3光子熵-量子机。结果表明,HUBO式优化提供了一种合理聚合推理碎片的方法,尤其在前沿模型尚未吸收基准内容的低污染领域优势明显。
原文摘要 · Abstract (English)
Large language models (LLMs) now solve a wide range of expert-level exams at or above human level, yet remain brittle on specialised, evidence-intensive domains such as law. On these tasks, errors arise not only from gaps in world knowledge but also from subtle distinctions between pieces of evidence and inconsistent use of supporting evidence. The most common aggregator over sampled chain-of-thought (CoT) traces, majority vote, returns the most popular answer regardless of whether its evidence is actually strongest. We propose to treat the selection of CoT reasoning fragments into a set of evidence as an explicit combinatorial optimisation problem, allowing well-supported but minority hypotheses to override noisy majorities, and to evaluate the approach on legal-reasoning benchmarks that are particularly sensitive to evidence quality. We introduce EP-HUBO (Evidence Pool Higher-Order Binary Optimisation), which generates multiple CoT traces with a small local model, parses fragments into per-hypothesis evidence pools, solves a higher-order unconstrained binary optimisation per pool with quality-derived weights (relevance, specificity, distinctiveness), and delegates a single adjudication call per question to a frontier model. We evaluate EP-HUBO on two evidence-intensive legal benchmarks using both simulated annealing on classical hardware and the Dirac-3 photonic entropy-quantum machine from Quantum Computing Inc. HUBO-style optimisation gives a principled way to aggregate reasoning fragments while preserving minority-but-correct hypotheses, and is most valuable in low-contamination domains where frontier models have not already absorbed the benchmark material.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。