用大模型自动发现文档中的语义结构，提升文本生成质量。
fLSA: Learning Semantic Structures in Document Collections Using Foundation Models
- 基于大模型迭代聚类并打标签，捕捉文档层级语义结构。
- 在故事生成等数据集上，标签信息量高于现有方法。
- 适合需要逻辑推理和结构化生成的场景，如数学解题。
人类可通过观察相似问题的解法，归纳出高层策略并应用于未见问题。我们能否利用大语言模型从示例文档中提取此类高层结构？本文提出fLSA，一种基于基础模型的潜在语义分析方法，通过文档级上下文迭代聚类与标记文档片段。这些标签可用于建模文档的潜在结构，并支持层次化文本采样。在故事写作、数学及多步推理数据集上的实验表明，fLSA的标签在重构原文时比现有打标方法更具信息量；当用于层次化采样时，其标签能更有效地引导生成空间，使正确解出现频率高于直接采样和现有方法。代码已开源：https://github.com/microsoft/fLSA。
原文摘要 · Abstract (English)
Humans can learn to solve new tasks by inducing high-level strategies from example solutions to similar problems and then adapting these strategies to solve unseen problems. Can we use large language models to induce such high-level structure from example documents or solutions? We introduce fLSA, a foundation-model-based Latent Semantic Analysis method that iteratively clusters and tags document segments based on document-level contexts. These tags can be used to model the latent structure of given documents and for hierarchical sampling of new texts. Our experiments on story writing, math, and multi-step reasoning datasets demonstrate that fLSA tags are more informative in reconstructing the original texts than existing tagging methods. Moreover, when used for hierarchical sampling, fLSA tags help expand the output space in the right directions that lead to correct solutions more often than direct sampling and hierarchical sampling with existing tagging methods. Code: https://github.com/microsoft/fLSA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。