用大模型自动生成查询变体，效果接近人工。
Can Generative LLMs Create Query Variants for Test Collections? An Exploratory Study
- 输入信息需求背景，让大模型生成查询变体。
- 生成的查询能覆盖71.1%的人工相关文档（前100名）。
- 适合需要快速构建测试集的研究者使用。
本文探索了大型语言模型（LLM）从信息需求背景中自动生成查询及其变体的可行性。给定一组以背景故事描述的信息需求，研究比较了模型生成的查询与人工生成查询的相似性。通过多种度量方法评估相似性，并分析不同查询集在构建测试集时对文档池的贡献。结果表明，大模型生成的查询虽未能完全复现人类查询的多样性，但能检索到高度重合的相关文档，在前100篇文档中最高达到71.1%的重叠率，展现出在测试集构建中的应用潜力。
原文摘要 · Abstract (English)
This paper explores the utility of a Large Language Model (LLM) to automatically generate queries and query variants from a description of an information need. Given a set of information needs described as backstories, we explore how similar the queries generated by the LLM are to those generated by humans. We quantify the similarity using different metrics and examine how the use of each set would contribute to document pooling when building test collections. Our results show potential in using LLMs to generate query variants. While they may not fully capture the wide variety of human-generated variants, they generate similar sets of relevant documents, reaching up to 71.1% overlap at a pool depth of 100.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。