PISCO实现16倍文档压缩，零预训练下保持高问答准确率。
PISCO: Pretty Simple Compression for Retrieval-Augmented Generation
- 通过序列级知识蒸馏实现无预训练压缩，仅需原始文档问答对。
- 在多种QA任务中压缩16倍，准确率损失仅0-3%。
- 可在单块A100上48小时完成7-10B模型微调，适合大规模部署。
检索增强生成（RAG）通过引入相关文档提升大语言模型性能，但面临推理成本高和上下文长度受限的问题。文档压缩是有效解决方案，但现有软压缩方法存在精度下降且需大量预训练。本文提出PISCO，一种无需预训练或标注数据的新方法，仅依赖文档问答的序列级知识蒸馏，实现16倍压缩率，跨多种RAG问答任务保持0-3%的极低精度损失。实验表明，PISCO在准确率上比现有压缩模型高出8%。该方法可在单张A100 GPU上48小时内完成7-10B模型的微调，兼具高效与可扩展性。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) pipelines enhance Large Language Models (LLMs) by retrieving relevant documents, but they face scalability issues due to high inference costs and limited context size. Document compression is a practical solution, but current soft compression methods suffer from accuracy losses and require extensive pretraining. In this paper, we introduce PISCO, a novel method that achieves a 16x compression rate with minimal accuracy loss (0-3%) across diverse RAG-based question-answering (QA) tasks. Unlike existing approaches, PISCO requires no pretraining or annotated data, relying solely on sequence-level knowledge distillation from document-based questions. With the ability to fine-tune a 7-10B LLM in 48 hours on a single A100 GPU, PISCO offers a highly efficient and scalable solution. We present comprehensive experiments showing that PISCO outperforms existing compression models by 8% in accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。