HLTCOE在TREC2024中融合多模型与翻译技术,提升跨语言信息检索与报告生成效果。
HLTCOE at TREC 2024 NeuCLIR Track
- 采用mT5重排序器与GPT-4重排序,结合文档翻译与得分融合策略。
- 使用翻译蒸馏、生成蒸馏及多语言蒸馏训练,提升跨语言匹配性能。
- 利用大模型生成摘要或整合事实,实现多任务报告自动生成。
HLTCOE团队在TREC 2024 NeuCLIR赛道中,采用PLAID(一种mT5重排序器)、GPT-4重排序、得分融合及文档翻译技术。PLAID引入多种模型与训练方法:翻译蒸馏(TD)利用英文MS MARCO查询-文档对训练mT5模型以评分翻译后的文档;生成蒸馏(GD)基于大模型生成的段落与查询进行训练;多语言翻译蒸馏(MTD)使用多语言版本的MS MARCO,探索数据批处理方式。报告生成方面,实验了系统组合策略:一组使用GPT-4o或Claude-3.5-Sonnet总结分解子问题的检索结果;另一组则用Claude-3.5-Sonnet验证并合并两模型输出;第三类使用GPT-4o与GPT-3.5-Turbo从检索文档中提取并分组相关事实,直接拼接成报告,并标注原文来源。团队提交了所有任务的运行结果,包括跨语言信息检索(CLIR)、多语言新闻检索(MLIR)、技术文档任务及报告生成任务。
原文摘要 · Abstract (English)
The HLTCOE team applied PLAID, an mT5 reranker, GPT-4 reranker, score fusion, and document translation to the TREC 2024 NeuCLIR track. For PLAID we included a variety of models and training techniques -- Translate Distill (TD), Generate Distill (GD) and multi-lingual translate-distill (MTD). TD uses scores from the mT5 model over English MS MARCO query-document pairs to learn how to score query-document pairs where the documents are translated to match the CLIR setting. GD follows TD but uses passages from the collection and queries generated by an LLM for training examples. MTD uses MS MARCO translated into multiple languages, allowing experiments on how to batch the data during training. Finally, for report generation we experimented with system combination over different runs. One family of systems used either GPT-4o or Claude-3.5-Sonnet to summarize the retrieved results from a series of decomposed sub-questions. Another system took the output from those two models and verified/combined them with Claude-3.5-Sonnet. The other family used GPT4o and GPT3.5Turbo to extract and group relevant facts from the retrieved documents based on the decomposed queries. The resulting submissions directly concatenate the grouped facts to form the report and their documents of origin as the citations. The team submitted runs to all NeuCLIR tasks: CLIR and MLIR news tasks as well as the technical documents task and the report generation task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。