LSP让大模型推理更可靠，规则执行更精准可追溯。
Logic Sketch Prompting (LSP): A Deterministic and Interpretable Prompting Method
- 用类型变量和规则验证器实现确定性推理
- 在药物合规任务中准确率最高达89%
- 适合医疗等需要可审计决策的场景
大型语言模型在自然语言推理上表现优异，但在需要严格遵守规则、确定性和可审计性的任务中仍不可靠。逻辑草图提示（LSP）是一种轻量级提示框架，引入了类型变量、确定性条件评估器和基于规则的验证器，生成可追踪且可重复的输出。我们在两个药理学逻辑合规任务上，将LSP与零样本提示、思维链提示和简洁提示对比，涵盖三个开源权重模型：Gemma 2、Mistral 和 Llama 3。在所有任务和模型中，LSP的准确率均达到0.83至0.89，F1分数为0.83至0.89，显著优于零样本提示（0.24至0.60）、简洁提示（0.16至0.30）和思维链提示（0.56至0.75）。麦克内马尔检验显示，几乎所有对比中LSP均有统计学显著优势（p < 0.01）。结果表明，LSP在不牺牲性能的前提下提升了确定性、可解释性和一致性，适用于临床、监管及安全关键决策支持系统。
原文摘要 · Abstract (English)
Large language models (LLMs) excel at natural language reasoning but remain unreliable on tasks requiring strict rule adherence, determinism, and auditability. Logic Sketch Prompting (LSP) is a lightweight prompting framework that introduces typed variables, deterministic condition evaluators, and a rule based validator that produces traceable and repeatable outputs. Using two pharmacologic logic compliance tasks, we benchmark LSP against zero shot prompting, chain of thought prompting, and concise prompting across three open weight models: Gemma 2, Mistral, and Llama 3. Across both tasks and all models, LSP consistently achieves the highest accuracy (0.83 to 0.89) and F1 score (0.83 to 0.89), substantially outperforming zero shot prompting (0.24 to 0.60), concise prompts (0.16 to 0.30), and chain of thought prompting (0.56 to 0.75). McNemar tests show statistically significant gains for LSP across nearly all comparisons (p < 0.01). These results demonstrate that LSP improves determinism, interpretability, and consistency without sacrificing performance, supporting its use in clinical, regulated, and safety critical decision support systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。