用解释引导判断，让小模型也能准确识别句子是否合语法。
Explain-then-Process: Using Grammar Prompting to Enhance Grammatical Acceptability Judgments
- 先让大模型解释语法规则，再将解释反馈给目标模型做判断。
- 在多语言数据集上，小模型准确率提升20%，与大模型差距缩小至5.8个百分点。
- 方法轻量通用，适合资源有限但需高精度语法判断的场景。
大型语言模型（LLM）能解释语法规则，但在判断句子是否合语法时往往无法应用这些规则。本文提出“语法提示”（grammar prompting）：先由大模型生成简洁的句法现象解释，再将该解释作为额外上下文输入目标模型（无论是大模型还是小型语言模型），以决定最小对比句中哪个更符合语法。在英语BLiMP、中文SLING和俄语RuBLiMP基准测试中，该提示方法在多种句法现象上显著优于强基线。将大模型的元语言解释反馈给目标模型，有效弥合了‘知道规则’与‘使用规则’之间的鸿沟。对于小型语言模型，仅使用语法提示即可将平均大模型与小模型的准确率差距缩小约20%；若结合思维链（chain-of-thought），差距进一步缩小至5.6个百分点（13.0个百分点 → 5.8个百分点），且成本极低。这一轻量、跨语言通用的提示机制，使低成本小模型在多语言环境下逼近前沿大模型性能。
原文摘要 · Abstract (English)
Large language models (LLMs) can explain grammatical rules, yet they often fail to apply those rules when judging sentence acceptability. We present "grammar prompting", an explain-then-process paradigm: a large LLM first produces a concise explanation of the relevant syntactic phenomenon, then that explanation is fed back as additional context to the target model -- either an LLM or a smaller language model (SLM) -- before deciding which sentence of a minimal pair is grammatical. On the English BLiMP, Chinese SLING, and Russian RuBLiMP benchmarks, this simple prompt design yields substantial improvements over strong baselines across many syntactic phenomena. Feeding an LLM's metalinguistic explanation back to the target model bridges the gap between knowing a rule and using it. On SLMs, grammar prompting alone trims the average LLM-SLM accuracy gap by about 20%, and when paired with chain-of-thought, by 56% (13.0 pp -> 5.8 pp), all at negligible cost. The lightweight, language-agnostic cue lets low-cost SLMs approach frontier-LLM performance in multilingual settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。