通过标签增强提升大模型在长文本中的问答能力
Tagging-Augmented Generation: Assisting Language Models in Finding Intricate Knowledge In Long Contexts
- 在上下文或提示中添加标签信息，无需修改原始文档
- 在32K token长文本上性能提升最高达17%
- 适合需要跨长文本推理的复杂问答任务
近期研究发现，当前主流大语言模型在处理长而复杂的上下文时，问答和推理能力存在显著局限。尽管检索增强生成（RAG）和分块重排序等方法试图缓解此问题，但对分块策略、嵌入方式和检索模型敏感，且依赖大量预处理与知识索引。本文提出轻量级数据增强策略Tagging-Augmented Generation（TAG），在不破坏原始文档完整性的情况下，显著提升大模型在长上下文场景下的表现。我们在两个高难度问答基准NoLima和NovelQA上验证该方法，结果显示：对上下文添加标签，或仅在提示中加入标签定义，即可带来稳定性能提升——在32K token上下文中最高提升17%，在多跳推理任务中复杂问答准确率提升2.9%。
原文摘要 · Abstract (English)
Recent investigations into effective context lengths of modern flagship large language models (LLMs) have revealed major limitations in effective question answering (QA) and reasoning over long and complex contexts for even the largest and most impressive cadre of models. While approaches like retrieval-augmented generation (RAG) and chunk-based re-ranking attempt to mitigate this issue, they are sensitive to chunking, embedding and retrieval strategies and models, and furthermore, rely on extensive pre-processing, knowledge acquisition and indexing steps. In this paper, we propose Tagging-Augmented Generation (TAG), a lightweight data augmentation strategy that boosts LLM performance in long-context scenarios, without degrading and altering the integrity and composition of retrieved documents. We validate our hypothesis by augmenting two challenging and directly relevant question-answering benchmarks -- NoLima and NovelQA -- and show that tagging the context or even just adding tag definitions into QA prompts leads to consistent performance gains over the baseline -- up to 17% for 32K token contexts, and 2.9% in complex reasoning question-answering for multi-hop queries requiring knowledge across a wide span of text. Additional details are available at https://sites.google.com/view/tag-emnlp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。