用少样本优化提升低资源场景下的幻觉检测准确率
Few-Shot Optimized Framework for Hallucination Detection in Resource-Limited NLP Systems
- 通过迭代提示工程优化弱标签生成，重构数据以适配指令模型
- 在SHROOM任务上达85.5%准确率，超越现有方法
- 适合低资源环境下的文本生成可靠性评估
文本生成中的幻觉检测在自然语言处理中仍是难题，常导致机器翻译和定义建模等应用输出不可靠。现有方法受限于数据稀缺与无标签数据集的不足，如SemEval-2024的SHROOM共享任务所示。本文提出一种新框架，引入DeepSeek少样本优化，通过迭代提示工程增强弱标签生成，并重构数据以对齐指令生成模型。基于优化后的标注数据，我们微调了Mistral-7B-Instruct-v0.3模型，使其在资源受限环境下准确检测幻觉。结合集成学习策略，该方法在测试集上达到85.5%的准确率，为SHROOM任务设立新基准。研究证明，数据重构、少样本优化与微调在构建可扩展、鲁棒的幻觉检测框架中有效。
原文摘要 · Abstract (English)
Hallucination detection in text generation remains an ongoing struggle for natural language processing (NLP) systems, frequently resulting in unreliable outputs in applications such as machine translation and definition modeling. Existing methods struggle with data scarcity and the limitations of unlabeled datasets, as highlighted by the SHROOM shared task at SemEval-2024. In this work, we propose a novel framework to address these challenges, introducing DeepSeek Few-shot optimization to enhance weak label generation through iterative prompt engineering. We achieved high-quality annotations that considerably enhanced the performance of downstream models by restructuring data to align with instruct generative models. We further fine-tuned the Mistral-7B-Instruct-v0.3 model on these optimized annotations, enabling it to accurately detect hallucinations in resource-limited settings. Combining this fine-tuned model with ensemble learning strategies, our approach achieved 85.5% accuracy on the test set, setting a new benchmark for the SHROOM task. This study demonstrates the effectiveness of data restructuring, few-shot optimization, and fine-tuning in building scalable and robust hallucination detection frameworks for resource-constrained NLP systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。