用轻量编码器检测回答是否基于上下文，速度快且准确。
Small Encoders Can Rival Large Decoders in Detecting Groundedness
- 用微调的RoBERTa等轻量编码器提前判断答案是否可信。
- 在数据集上准确率媲美Llama3 8B和GPT4o，延迟降低数量级。
- 适合需要快速、低成本推理的场景，如实时问答系统。
将外部上下文引入大语言模型（LLM）能显著提升自然语言处理任务表现。然而，当上下文信息不足时，LLM常依赖内部知识进行无根据推测，导致回答不真实。为确保事实一致性与可信度，需保证生成内容严格基于上下文，即实现‘可溯源性’。本研究聚焦于在昂贵的LLM答句生成前，提前检测查询是否可由给定文档支撑。该机制可大幅降低推理时间与资源消耗。实验表明，经精心数据集微调的轻量级编码器（如RoBERTa、NomicBERT），在可溯源性检测任务中表现可与SOTA LLM（如Llama3 8B、GPT4o）相当，同时推理延迟下降数个数量级。代码已开源：https://github.com/chandarlab/Hallucinate-less。
原文摘要 · Abstract (English)
Augmenting large language models (LLMs) with external context significantly improves their performance in natural language processing (NLP) tasks. However, LLMs struggle to answer queries reliably when the provided context lacks information, often resorting to ungrounded speculation or internal knowledge. Groundedness - generating responses strictly supported by the context - is essential for ensuring factual consistency and trustworthiness. This study focuses on detecting whether a given query is grounded in a document provided in context before the costly answer generation by LLMs. Such a detection mechanism can significantly reduce both inference time and resource consumption. We show that lightweight, task specific encoder models such as RoBERTa and NomicBERT, fine-tuned on curated datasets, can achieve accuracy comparable to state-of-the-art LLMs, such as Llama3 8B and GPT4o, in groundedness detection while reducing inference latency by orders of magnitude. The code is available at : https://github.com/chandarlab/Hallucinate-less
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。