用逻辑规则+大模型实现可解释、抗干扰的政策问答
Policy-as-logic for robust reasoning over rules
- 将政策转为形式化逻辑，大模型提取事实，求解器推理
- 相比提示词/代码方法，准确率更高，耗token减少约90%
- 适合需要可审计、高鲁棒性的政策类问答场景
在生成式AI的实际应用中，如税务规则或航空行李限额，自然语言查询的回复必须遵循书面政策。我们提出一种混合符号方法：将政策表示为形式逻辑，在推理时利用语言模型提取事实以确定谓词，再通过答案集求解器进行推理，使回复具备可解释性、可审计性，并且如我们所示，在输入扰动下仍保持准确与鲁棒。具体而言，这种提取与推理分离的策略在多数情况下优于政策作为提示词和政策作为代码的方法，同时将令牌使用量减少了约10倍。结果表明，结合结构化推理与符号求解器，能有效支持基于客观标准的稳健决策。
原文摘要 · Abstract (English)
In many practical applications of generative AI systems, from tax rules to airline baggage allowance, responses to natural language queries must respect written policies or rules. We present a hybrid symbolic approach that expresses policies in formal logic and at inference time exploits the representation power of language models for fact extraction to ground predicates, and an answer set solver for reasoning such that responses are interpretable, auditable, and as we show, accurate and robust under input perturbations. Specifically, we show this separation of extraction and reasoning steps outperforms policy-as-prompt and policy-as-code methods in most cases with ~10x reduction in token usage. The results point to the value of structured reasoning and symbolic solvers in conjunction with generative models to make robust decisions involving objective criteria.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。