让大模型像人一样表达不确定,提升可信度。
Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing
- 模仿人类说话方式表达不确定,增强信任感
- 发现现有模型在不确定性表达上存在明显偏差
- 适合关注人机协作可信度的研究者
随着用户越来越多地与大型语言模型(LLMs)交互,这些模型常常在准确性存疑时仍表现出过度自信,损害其可信度和合法性。因此,需要让语言模型能够传达自身信心以促进人机协作并减少潜在危害。言语化不确定性是指用语言方式表达信心,这种机制与基于语言的界面天然契合。然而,当前自然语言处理(NLP)研究普遍忽视了人类不确定性沟通中的细微差别以及影响人机互动的偏见。本文主张采用‘拟人化不确定性’原则,即通过模仿人类的语言行为来实现直观且可信的不确定性表达。我们系统梳理了人类不确定性沟通的研究,综述了当前NLP领域的进展,并进行了额外分析,揭示了目前尚未充分探索的言语化不确定性中的偏见。最后,我们指出人机不确定性交流的独特因素,并展望未来实现拟人化不确定性的研究方向。
原文摘要 · Abstract (English)
Human users increasingly communicate with large language models (LLMs), but LLMs suffer from frequent overconfidence in their output, even when its accuracy is questionable, which undermines their trustworthiness and perceived legitimacy. Therefore, there is a need for language models to signal their confidence in order to reap the benefits of human-machine collaboration and mitigate potential harms. Verbalized uncertainty is the expression of confidence with linguistic means, an approach that integrates perfectly into language-based interfaces. Most recent research in natural language processing (NLP) overlooks the nuances surrounding human uncertainty communication and the biases that influence the communication of and with machines. We argue for anthropomimetic uncertainty, the principle that intuitive and trustworthy uncertainty communication requires a degree of imitation of human linguistic behaviors. We present a thorough overview of the research in human uncertainty communication, survey ongoing research in NLP, and perform additional analyses to demonstrate so-far underexplored biases in verbalized uncertainty. We conclude by pointing out unique factors in human-machine uncertainty and outlining future research directions towards implementing anthropomimetic uncertainty.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。