用大模型为复杂问题的子问题打标签，提升检索模型性能
Augmenting Researchy Questions with Sub-question Judgments
- 用Llama3.3 70B为10万条研究型问题生成子问题相关性标签
- 补充了原始数据中缺失的子问题与文档关联信息
- 适合做复杂查询检索、多跳问答和大模型评估的研究者
Researchy Questions 数据集包含约10万条需要多方面信息的复杂查询，每条查询都配有GPT-4生成的子问题。尽管该数据集提供了用户点击的文档信息，但缺乏子问题与相关文档之间的标注。本文使用Llama3.3 70B模型为每个子问题生成相关性判断标签，扩充了该数据集。这些新增的子问题标签旨在支持更精准的复杂信息需求检索模型训练。
原文摘要 · Abstract (English)
The Researchy Questions dataset provides about 100k question queries with complex information needs that require retrieving information about several aspects of a topic. Each query in ResearchyQuestions is associated with sub-questions that were produced by prompting GPT-4. While ResearchyQuestions contains labels indicating what documents were clicked after issuing the query, there are no associations in the dataset between sub-questions and relevant documents. In this work, we augment the Researchy Questions dataset with LLM-judged labels for each sub-question using a Llama3.3 70B model. We intend these sub-question labels to serve as a resource for training retrieval models that better support complex information needs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。