研究提示语和回复语言如何影响大模型生成内容，发现语言选择会显著改变输出长度。
Investigating the Influence of Prompt and Response Languages on LLM Content Generation
- 对比四种语言组合，分析提示语与回复语言对生成文本的影响
- 用挪威语提示时，英文回复长度减少约37%，英文提示时挪威语回复减少41%
- 尽管长度变化明显，语义相似度仍高，适合多语言系统设计参考
本研究考察了提示语和回复语言对大语言模型行为的影响。使用五个模型，在四种语言条件下评估了68个非翻译类问题的回答：英语→英语、英语→挪威语、挪威语→挪威语、挪威语→英语。去除拒绝项后，数据集包含1348条响应。通过Cohen's d衡量长度差异，使用LabSE余弦相似度评估语义保真度，采用原始与软化Jaccard指数衡量跨语言关键词重叠。结果显示，提示语语言对响应长度有显著影响：使用英文输出时，挪威语提示使响应长度缩短约37%；使用挪威语输出时，英文提示使响应长度缩短约41%。最大跨语言差异表现为词数减少但标记数减少较少，反映出分词器差异。尽管长度变化显著，语义相似度保持较高水平，软化Jaccard揭示了原始Jaccard未捕捉到的实质性概念重叠。不同模型间效应量存在差异，表明模型异质性。提示语语言并非中立，会系统性地影响输出长度与词汇实现，对多语言提示设计具有重要意义。
原文摘要 · Abstract (English)
This study examines how prompt and response language influence the behavior of large language models. Using five models, we evaluated answers to 68 non translation questions across four language conditions: English to English, English to Norwegian, Norwegian to Norwegian, and Norwegian to English. After removing refused items, the dataset contains 1348 responses. We measure length differences with Cohen d, semantic fidelity with LabSE cosine similarity, and cross lingual keyword overlap with both raw and soft Jaccard. Prompt language has a strong effect on response length. With English output, Norwegian prompts shorten responses by about thirty seven percent. With Norwegian output, English prompts shorten responses by about forty one percent. The largest cross lingual contrast shows a reduction in word count but a smaller reduction in tokens, reflecting tokenizer differences. Despite variation in length, semantic similarity remains high, and soft Jaccard reveals substantial conceptual overlap that raw Jaccard does not capture. Effect sizes vary across models, indicating heterogeneity. Prompt language is not neutral and systematically shapes output length and lexical realization, with implications for multilingual prompt design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。