RAGViz可可视化大模型生成时对检索文档的注意力，帮助诊断和优化RAG系统。
RAGViz: Diagnose and Visualize Retrieval-Augmented Generation
- 通过词元与文档级注意力图，直观展示生成过程中的关注点。
- 支持添加/移除上下文文档后生成结果对比，定位关键信息影响。
- 开源工具，兼容HuggingFace模型，5秒内完成中等规模查询。
检索增强生成（RAG）将领域特定知识融入大语言模型以支撑回答生成。现有RAG系统缺乏对检索文档及模型注意力的可定制化可视性。我们提出RAGViz，一个用于诊断和可视化生成词元在检索文档中注意力的工具。该工具具备内置用户界面、检索索引和大语言模型（LLM）核心，提供两大功能：(1) 词元与文档级别的注意力可视化；(2) 在添加或移除上下文文档后进行生成结果对比。作为开源套件，RAGViz可轻松集成自定义嵌入模型和HuggingFace支持的LLM核心。借助混合近似最近邻（ANN）索引、内存高效的LLM推理工具及自定义上下文片段方法，RAGViz在中等配置GPU节点上实现约5秒的平均查询时间。代码已公开于https://github.com/cxcscmu/RAGViz，演示视频见https://youtu.be/cTAbuTu6ur4。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) combines knowledge from domain-specific sources into large language models to ground answer generation. Current RAG systems lack customizable visibility on the context documents and the model's attentiveness towards such documents. We propose RAGViz, a RAG diagnosis tool that visualizes the attentiveness of the generated tokens in retrieved documents. With a built-in user interface, retrieval index, and Large Language Model (LLM) backbone, RAGViz provides two main functionalities: (1) token and document-level attention visualization, and (2) generation comparison upon context document addition and removal. As an open-source toolkit, RAGViz can be easily hosted with a custom embedding model and HuggingFace-supported LLM backbone. Using a hybrid ANN (Approximate Nearest Neighbor) index, memory-efficient LLM inference tool, and custom context snippet method, RAGViz operates efficiently with a median query time of about 5 seconds on a moderate GPU node. Our code is available at https://github.com/cxcscmu/RAGViz. A demo video of RAGViz can be found at https://youtu.be/cTAbuTu6ur4.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。