不依赖GPU，用普通电脑也能检测AI幻觉，但效果因任务而异。
How Far Can You Get Without a GPU? A Systematic Benchmark of Lightweight Hallucination Detection Across Question Answering, Dialogue, and Summarisation
- 仅用公开模型和CPU运行，测试五种轻量级幻觉检测方法
- 问答任务中集成方法表现最佳（F1=0.792），摘要任务几乎失效
- 结果揭示无GPU场景下检测能力的边界，指导资源受限者选型
幻觉检测对可信AI部署至关重要。现有最准确的方法依赖于高算力推理、专有API或生成模型的白盒访问，使资源有限的研究者难以使用。本文系统评估了仅依赖公开模型、可在普通笔记本CPU上运行的轻量级检测方法在三大任务上的表现：问答、对话和摘要。我们对比了五种方法：ROUGE-L、语义相似度、BERTScore、基于FEVER训练的DeBERTa的NLI检测器，以及相似度与NLI的分数级集成。在每个任务的2000个测试实例上进行校准与评估。结果发现，没有单一方法在所有任务中占优，性能高度依赖任务类型。集成方法在问答任务中表现最优（F1=0.792，AUC-ROC=0.873），NLI检测器在对话任务中领先（AUC-ROC=0.713），而所有方法在摘要任务中均退化至近随机水平（AUC-ROC介于0.469至0.574之间）。该任务依赖性及摘要任务的系统性失败，划定了无GPU条件下幻觉检测的实际边界，为计算受限场景下的方法选择提供了实用指导。所有实验均在标准笔记本CPU上完成。
原文摘要 · Abstract (English)
Hallucination detection has become a pressing requirement for trustworthy AI deployment at scale. The most accurate detection methods depend on GPU-intensive inference, proprietary API calls, or white-box access to the generating model. This puts them out of reach for resource-constrained researchers and practitioners. In this paper, we explore a practical alternative: how well can hallucination detection perform using only lightweight, CPU-feasible methods built on publicly available models? We systematically benchmark five such methods: ROUGE-L, semantic similarity, BERTScore, a Natural Language Inference (NLI) detector based on a FEVER-trained DeBERTa model, and a score-level ensemble of similarity and NLI. We evaluate them across all three tasks of the HaluEval benchmark: question answering (QA), dialogue, and summarisation. We calibrate each method on a held-out validation split and evaluate it on 2,000 test instances per task. We find that no single method dominates and performance is highly task-dependent. The ensemble performs best on QA (F1 = 0.792, AUC-ROC = 0.873), the NLI detector leads on dialogue (AUC-ROC = 0.713), and all five methods degrade to near-random performance on summarisation (AUC-ROC between 0.469 and 0.574). This task-dependence and the systematic failure on summarisation map the practical frontier of GPU-free hallucination detection. They give practical guidance for method selection under computational constraints. All experiments run on a standard laptop CPU using public models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。