用小模型提前消解大模型提示词的语义歧义,提升推理准确率。
Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt

- 用小模型检测并消除提示词中的语义风险与矛盾
- 在多个基准上提升推理性能2.5点,成本仅0.02美元
- 适合需要高精度推理但提示不规范的场景
大型语言模型(LLM)在复杂推理任务中表现优异,但其性能高度依赖用户输入提示的明确性。自然语言提示常缺乏语法规范,导致语义模糊,引发多种解释,使模型难以选择正确推理路径。现有方法在推理过程中进行查询编辑,未能根治歧义问题。为此,本文提出一种预推理阶段的提示优化机制,通过显式消解提示语义歧义:识别提示中的语义风险,多角度验证一致性,解决语义冲突,并将修正后的信息结构化为清晰输入。该方法使模型对关键语义标记的关注更集中。我们采用小语言模型(SLMs)执行消歧,利用其高效计算优势。在多个基准上的实验表明,该方法提升推理性能2.5点,成本仅为0.02美元。研究证明,显式消歧是一种有效且不干扰大模型内部机制的提示优化方法。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly utilized in various complex reasoning tasks due to their excellent instruction following capability. However, the model's performance is highly dependent on the open-ended characteristics of the users' input prompt. Natural prompts often do not follow proper syntactic rules, which creates ambiguous queries that yield multiple interpretations. Such ambiguous prompts confuse the model in choosing the correct reasoning paths to answer questions. Prior works address this challenge by applying query editing during the LLM inference process without explicitly solving the root cause of the ambiguity. To address this limitation, we propose a pre-inference prompt optimization mechanism via explicit prompt disambiguation. Particularly, we identify semantic risks in the prompt, check their multi-perspective consistency, and resolve any semantic conflicts that arise. Finally, we organize the resolved ambiguities in a logically structured manner as a clean input to the LLM. By explicitly resolving semantic ambiguity, our method can produce a more focused attention distribution to the semantically essential tokens. We also leverage small language models (SLMs) as the main executor of prompt disambiguation to benefit from their efficient computation. Through comprehensive experiments on multiple benchmarks, we demonstrate that our method improves reasoning performance by 2.5 points at a cost of only \$0.02. Our study promotes explicit prompt disambiguation as an effective prompt optimization method without disturbing the internal mechanism of LLM inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。