FreeChunker让检索生成系统灵活组合句子，提升查询适应性与效率。
FreeChunker: A Cross-Granularity Chunking Framework
- 以句子为基本单元，动态组合而非固定分块
- 在LongBench V2上检索性能与速度均优于现有方法
- 适合需要灵活响应复杂查询的RAG应用场景
分块策略显著影响检索增强生成（RAG）系统的有效性。现有方法依赖静态边界识别，局限于固定粒度，难以适应多样查询需求。本文提出FreeChunker，一种跨粒度编码框架，将句子视为基本单元，从静态分块转向支持任意句子组合的灵活检索。这一范式转变不仅避免了语义边界检测的计算开销，还提升了对复杂查询的适应能力。在LongBench V2上的实验表明，FreeChunker在检索性能和时间效率方面均显著优于现有分块方法。预训练模型与代码已开源：https://github.com/mazehart/FreeChunker。
原文摘要 · Abstract (English)
Chunking strategies significantly impact the effectiveness of Retrieval-Augmented Generation (RAG) systems. Existing methods operate within fixed-granularity paradigms that rely on static boundary identification, limiting their adaptability to diverse query requirements. This paper presents FreeChunker, a Cross-Granularity Encoding Framework that fundamentally transforms the traditional chunking paradigm: the framework treats sentences as atomic units and shifts from static chunk segmentation to flexible retrieval supporting arbitrary sentence combinations. This paradigm shift not only significantly avoids the computational overhead required for semantic boundary detection, but also enhances adaptability to complex queries. Experimental evaluation on LongBench V2 demonstrates that FreeChunker possesses significant advantages in both retrieval performance and time efficiency compared to existing chunking methods. The pre-trained models and codes are available at https://github.com/mazehart/FreeChunker.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。