用大模型提升技术写作中参考文献的准确生成与引用。
LLM-Ref: Enhancing Reference Handling in Technical Writing with Large Language Models
- 直接从段落级文本检索并生成内容，支持精准引用。
- 迭代生成机制有效处理长上下文，提升输出质量。
- 在Ragas评分上提升3.25至6.26倍，显著改善准确性与相关性。
大型语言模型在数据合成方面表现优异，但在特定领域任务中可能不够准确，而检索增强生成（RAG）系统通过利用用户提供的数据来缓解这一问题。然而，RAG系统在检索和生成阶段均需优化，这会影响最终输出质量。本文提出LLM-Ref，一款面向多源文档的技术写作辅助工具，具备更强的参考文献合成与管理能力。与传统基于分块和索引的RAG系统不同，本工具直接从文本段落中检索并生成内容，便于从生成结果中直接提取引用信息，具有独特优势。此外，该工具采用迭代响应生成机制，有效应对语言模型上下文长度限制。相比基线RAG系统，本方法在Ragas评分上实现3.25×至6.26×的提升，该指标综合评估了生成内容的准确性、相关性和情境适切性，表明本方法显著提升了写作辅助工具的生成质量。
原文摘要 · Abstract (English)
Large Language Models (LLMs) excel in data synthesis but can be inaccurate in domain-specific tasks, which retrieval-augmented generation (RAG) systems address by leveraging user-provided data. However, RAGs require optimization in both retrieval and generation stages, which can affect output quality. In this paper, we present LLM-Ref, a writing assistant tool that aids researchers in writing articles from multiple source documents with enhanced reference synthesis and handling capabilities. Unlike traditional RAG systems that use chunking and indexing, our tool retrieves and generates content directly from text paragraphs. This method facilitates direct reference extraction from the generated outputs, a feature unique to our tool. Additionally, our tool employs iterative response generation, effectively managing lengthy contexts within the language model's constraints. Compared to baseline RAG-based systems, our approach achieves a $3.25\times$ to $6.26\times$ increase in Ragas score, a comprehensive metric that provides a holistic view of a RAG system's ability to produce accurate, relevant, and contextually appropriate responses. This improvement shows our method enhances the accuracy and contextual relevance of writing assistance tools.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。