通过几何去噪提升模型对知识边界的判断精度，减少幻觉。
Purging the Gray Zone: Latent-Geometric Denoising for Precise Knowledge Boundary Awareness
- 在隐空间中利用几何距离识别模糊区域，指导弃权训练。
- 在多个数据集上显著降低幻觉率，提升外分布泛化能力。
- 适合需要高可信度的问答系统与安全敏感场景使用。
大型语言模型常因无法准确感知自身知识边界而产生幻觉。现有弃权微调方法通常直接依据响应准确率划分数据集，导致模型在决策边界附近遭受严重标签噪声，进而引发高弃权率或幻觉。本文从隐空间表示视角揭示：决策超平面附近的‘灰区’内，内部信念模糊是性能瓶颈的核心。基于此，提出GeoDe（Geometric Denoising）框架，通过线性探测构建真值超平面，并以几何距离作为弃权决策的置信度信号，实现对模糊样本的过滤，同时保留高质量微调信号。在Llama3、Qwen3等多模型及TriviaQA、NQ、SciQ、SimpleQA等数据集上的实验表明，GeoDe显著提升模型真实性，并在分布外（OOD）场景中表现出强泛化能力。代码已公开于https://github.com/Notbesidemoon/GeoDe。
原文摘要 · Abstract (English)
Large language models (LLMs) often exhibit hallucinations due to their inability to accurately perceive their own knowledge boundaries. Existing abstention fine-tuning methods typically partition datasets directly based on response accuracy, causing models to suffer from severe label noise near the decision boundaries and consequently exhibit high rates of abstentions or hallucinations. This paper adopts a latent space representation perspective, revealing a "gray zone" near the decision hyperplane where internal belief ambiguity constitutes the core performance bottleneck. Based on this insight, we propose the **GeoDe** (**Geo**metric **De**noising) framework for abstention fine-tuning. This method constructs a truth hyperplane using linear probes and performs "geometric denoising" by employing geometric distance as a confidence signal for abstention decisions. This approach filters out ambiguous boundary samples while retaining high-fidelity signals for fine-tuning. Experiments across multiple models (Llama3, Qwen3) and benchmark datasets (TriviaQA, NQ, SciQ, SimpleQA) demonstrate that GeoDe significantly enhances model truthfulness and demonstrates strong generalization in out-of-distribution (OOD) scenarios. Code is available at https://github.com/Notbesidemoon/GeoDe.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。