让大模型学会区分问题类型，回答更准确
Are LLMs Aware that Some Questions are not Open-ended?
- 根据问题特征动态调整生成温度，提升对问题类型的感知
- 在事实类问题上显著减少幻觉，提升回答确定性
- 无需人工调参，适合希望提升回答质量的开发者
大型语言模型在多种场景下展现出强大的问答能力。然而，当面对不同类型的问题时，模型是否意识到某些问题有固定答案、需确定性回应，而另一些则可自由发挥？我们称此为模型的问题意识。缺乏问题意识会导致模型对封闭式问题过于随意，或对开放式问题过于刻板。本文首先评估了大模型的问题意识，实验发现其在事实知识等特定领域存在意识缺失，引发生成幻觉。为此，我们提出问题意识温度采样（QuATS）方法，通过问题特征自适应调整输出分布，实现无需人工调参的自动优化，在多个基准测试中一致提升模型表现。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown the impressive capability of answering questions in a wide range of scenarios. However, when LLMs face different types of questions, it is worth exploring whether LLMs are aware that some questions have limited answers and need to respond more deterministically but some do not. We refer to this as question awareness of LLMs. The lack of question awareness in LLMs leads to two phenomena that LLMs are: (1) too casual to answer non-open-ended questions or (2) too boring to answer open-ended questions. In this paper, we first evaluate the question awareness in LLMs. The experimental results show that LLMs have the issues of lacking awareness of questions in certain domains, e.g. factual knowledge, resulting in hallucinations during the generation. To mitigate these, we propose a method called Question Awareness Temperature Sampling (QuATS). This method enhances the question awareness of LLMs by adaptively adjusting the output distributions based on question features. The automatic adjustment in QuATS eliminates the need for manual temperature tuning in text generation and consistently improves model performance in various benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。