arXiv:2607.18232cs.CL2026-07ACL

研究用户如何表达信念,影响大模型是否听从上下文。

It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief

论文配图:It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief
图 1 · 摘自论文原文
  • 构建四维语言框架,分析信念表达的17种类型
  • 发现大模型和指令微调模型更少遵循上下文
  • 识别出更具说服力的表达方式,适合提示工程优化

用户常向大语言模型(LLMs)表达个人观点。在某些情境下,模型应接受这些信念为真;在另一些情境下,则应坚持自身先验知识。值得注意的是,用户表达信念(EoBs)的语言形式多样——如预设、证据标记、确定性词语或语气差异——可能对模型产生不同说服力。本文提出一个基于语言学的四维分类体系:形式、证据性、认识立场与语气,涵盖17种细粒度类型。通过将这些表达与世界知识事实配对,生成可控的EoB-查询对,以隔离语言变化的影响。使用该基准评估了16个不同架构(Llama3、Qwen3、Gemma3)、规模(1B–30B参数)和训练阶段(基础模型与指令微调模型)的LLM。结果显示,模型行为在各维度上存在显著差异,例如大模型和指令微调模型更倾向于不遵循上下文。进一步识别出若干在统计上显著更有效的表达方式。研究揭示了语言框架如何系统性影响模型对上下文的整合,对提示工程与模型鲁棒性具有重要启示。

原文摘要 · Abstract (English)

Users frequently express their beliefs to large language models (LLMs). In some situations, the LLM should accept these contextual beliefs as true. In others, they should stick to their prior knowledge. Notably, users' expressions of belief (EoBs) can take linguistically diverse forms - using presuppositions, evidential and certainty markers, or varied tones - each of which may have a different persuasiveness over the LLMs. We introduce a typology to systematically evaluate how different EoBs affect whether models follow context versus prior knowledge. The typology is grounded in four linguistically motivated dimensions: form, evidentiality, epistemic stance, and tone, spanning 17 fine-grained types. By pairing these EoBs with world knowledge facts, we generate controlled EoB-query pairs that isolate the effect of linguistic variation. Using this benchmark, we evaluate 16 LLMs that differ in architecture (Llama3, Qwen3, Gemma3), scale (1B-30B parameters), and training stages (base vs instruct). We identify meaningful variations in response behavior across these axes, e.g., that bigger models and instruction models tend to be less context-following than smaller models and base models. We further identify specific EoBs that statistically significantly persuade LMs more consistently than others. Our work reveals systematic patterns in how linguistic framing affects LLM context integration, with implications for prompt engineering and model robustness.

大模型信念表达提示工程语言学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。