让大模型学会推断言外之意,靠自我强化的反事实推理。
PragReST: Self-Reinforcing Counterfactual Reasoning for Pragmatic Language Understanding

- 构建反事实推理链,通过自监督训练提升模型语用理解能力。
- 在4个语用基准上,相比基线模型准确率提升5.37%至5.50%。
- 适合需要精准理解隐含意义的场景,如对话系统、智能客服。
自然语言理解常依赖于未明说的隐含意义,需借助语用推理。尽管大语言模型在数学和逻辑推理上表现良好,但在语用推断上仍倾向于字面理解。为此,我们提出PragReST,一种无需人工标注或强教师模型蒸馏的自监督框架,通过构建语用问答数据、生成反事实推理轨迹,并结合监督微调与强化学习,使模型内化推理过程。在四个语用基准(PragMega、Ludwig、MetoQA、AltPrag)上,PragReST优于基线模型及非反事实变体。在准确率基准上,对Qwen3-8B和Qwen3-14B分别提升5.37%和5.50%绝对值。错误分析与消融实验表明,反事实推理至关重要:主要减少因未对比实际话语与可能替代表达而产生的错误;移除该机制会显著降低性能。此外,训练过程保持了模型在通用知识与数学推理任务上的域外表现。
原文摘要 · Abstract (English)
Natural language understanding often depends on meanings that are implied rather than explicitly stated, requiring pragmatic reasoning. Despite strong performance on math and logical reasoning, large language models (LLMs) still struggle with making pragmatic inferences, often choosing literal interpretations. To improve LLM pragmatic reasoning, we introduce PragReST, a self-supervised framework that constructs pragmatic QA data, generates counterfactual reasoning traces, and trains models to internalize them through supervised fine-tuning and reinforcement learning, without human-labeled training data or distillation from a stronger teacher. Across four pragmatic benchmarks (PragMega, Ludwig, MetoQA, and AltPrag), PragReST improves over backbone models, task-specific pragmatic tuning baselines, and non-counterfactual variants of the same pipeline. On accuracy-based benchmarks, PragReST improves over the instruct backbone by 5.37 and 5.50% (absolute) for Qwen3-8B and Qwen3-14B, respectively. Our error analysis and ablations underscore the importance of counterfactual reasoning: PragReST primarily reduces errors caused by failures to contrast observed utterances with plausible alternatives, and removing counterfactual reasoning substantially reduces performance. Moreover, our training preserves out-of-domain performance on general-knowledge and mathematical reasoning benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。