用检索聚类构建知识边界,让长文生成更真实可信。
ConvergeWriter: Data-Driven Bottom-Up Article Construction
- 先检索再聚类,用数据驱动确定知识范围
- 在14B和32B模型上表现优于或媲美顶尖方法
- 适合对事实准确性和结构完整要求高的场景
大型语言模型在文本生成方面表现出色,但基于外部知识库生成长篇、事实性文档仍面临挑战。现有自上而下的方法先生成假设或大纲,再检索证据,常导致计划与知识脱节,引发内容碎片化和事实错误。为此,我们提出一种新型自下而上的数据驱动框架,采用‘先检索知识,再聚类定结构’策略,在生成规划前先建立源语料的知识边界。具体而言,通过从知识库中进行迭代式全面检索,并使用无监督聚类算法将检索到的文档划分为不同的‘知识簇’。这些簇构成客观、数据驱动的基础,直接指导后续层次化大纲与正文生成。该过程确保生成内容严格受限于且可追溯至原始材料,主动适应知识库的有限范围,从根本上降低幻觉风险。在14B和32B参数模型上的实验结果表明,该方法性能达到或超过当前最佳基线,尤其在需高保真度与结构连贯性的知识约束场景中具有显著优势。本研究为生成可靠、结构化的长篇文档提供有效范式,推动大模型在高风险、知识密集型领域应用的发展。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown remarkable prowess in text generation, yet producing long-form, factual documents grounded in extensive external knowledge bases remains a significant challenge. Existing "top-down" methods, which first generate a hypothesis or outline and then retrieve evidence, often suffer from a disconnect between the model's plan and the available knowledge, leading to content fragmentation and factual inaccuracies. To address these limitations, we propose a novel "bottom-up," data-driven framework that inverts the conventional generation pipeline. Our approach is predicated on a "Retrieval-First for Knowledge, Clustering for Structure" strategy, which first establishes the "knowledge boundaries" of the source corpus before any generative planning occurs. Specifically, we perform exhaustive iterative retrieval from the knowledge base and then employ an unsupervised clustering algorithm to organize the retrieved documents into distinct "knowledge clusters." These clusters form an objective, data-driven foundation that directly guides the subsequent generation of a hierarchical outline and the final document content. This bottom-up process ensures that the generated text is strictly constrained by and fully traceable to the source material, proactively adapting to the finite scope of the knowledge base and fundamentally mitigating the risk of hallucination. Experimental results on both 14B and 32B parameter models demonstrate that our method achieves performance comparable to or exceeding state-of-the-art baselines, and is expected to demonstrate unique advantages in knowledge-constrained scenarios that demand high fidelity and structural coherence. Our work presents an effective paradigm for generating reliable, structured, long-form documents, paving the way for more robust LLM applications in high-stakes, knowledge-intensive domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。