用无意义符号替换提示中的偏见词，能提升大模型逻辑推理能力
Meaningless is better: hashing bias-inducing words in LLM prompts improves performance in logical reasoning and statistical learning
- 用哈希符号替代提示中可能引发偏见的词汇
- 在490个提示上测试，显著降低认知偏差错误率
- 对不同模型和输入格式均有效，适合提升推理任务性能
本文提出一种新方法「哈希」，将大语言模型（LLM）提示中可能引发偏见的词汇替换为无意义的哈希标识符，以减少认知偏见和对外部知识的依赖。该方法在总计490个提示的三组实验中进行验证，涵盖LLama、ChatGPT、Copilot、Gemini和Mixtral等模型。卡方检验显示，所有测试场景均有显著改进：第一项实验中，修改版“Linda”问题的谬误率下降；第二项实验中，频繁项集提取任务表现提升；第三项实验表明，即使将Linda问题以表格形式呈现，哈希仍有效，说明该方法适用于多种输入形式。整体上，该方法有助于减少偏见并更好整合外部知识。尽管偏见减少，但幻觉率在不同模型间变化不一。结果表明，屏蔽偏见相关词汇可提升大模型性能，但效果因模型与任务而异。
原文摘要 · Abstract (English)
This paper introduces a novel method, referred to as "hashing", which involves masking potentially bias-inducing words in large language models (LLMs) with hash-like meaningless identifiers to reduce cognitive biases and reliance on external knowledge. The method was tested across three sets of experiments involving a total of 490 prompts. Statistical analysis using chi-square tests showed significant improvements in all tested scenarios, which covered LLama, ChatGPT, Copilot, Gemini and Mixtral models. In the first experiment, hashing decreased the fallacy rate in a modified version of the "Linda" problem aimed at evaluating susceptibility to cognitive biases. In the second experiment, it improved LLM results on the frequent itemset extraction task. In the third experiment, we found hashing is also effective when the Linda problem is presented in a tabular format rather than text, indicating that the technique works across various input representations. Overall, the method was shown to improve bias reduction and incorporation of external knowledge. Despite bias reduction, hallucination rates were inconsistently reduced across types of LLM models. These findings suggest that masking bias-inducing terms can improve LLM performance, although its effectiveness is model- and task-dependent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。