比较三种文本切分方法在学术论文问答中的表现
Evaluating Chunking Strategies for Retrieval-Augmented Generation on Academic Texts
- 用聚类语义切分尝试提升检索质量
- 固定大小切分在多数情况下表现更好
- 适合评估学术文本问答系统的优化策略
检索增强生成（RAG）系统利用大语言模型的问答能力获取其参数之外的信息。本文在长篇结构化学术论文上，采用RAGAs框架评估了基于聚类的语义切分、固定大小切分和递归切分的效果。结果表明，基于忠实度的RAGAs评估可靠性有限；针对固定问题与文档特定问题的表现差异显著，可能与文档格式和预处理有关。在测试配置下，聚类语义切分未优于更简单的切分策略。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems use the question-answering capabilities of Large Language Models (LLMs) to access information outside their parameters. We evaluate if cluster-based semantic chunking improves retrieval and answer quality compared to fixed-size and recursive chunking evaluating on long, structured academic theses using the Retrieval Augmented Generation Assessment (RAGAs) framework. RAGAs based faithfulness shows limited reliability in this setup. Performance on fixed versus document specific questions varied substantially, likely related to the formatting of documents and preprocessing. Under the tested configuration, cluster-based chunking did not outperform simpler strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。