打造统一工具箱，让大模型改写查询更可复现、易比较。
QueryGym: A Toolkit for Reproducible LLM-Based Query Reformulation
- 提供统一Python接口，支持多种大模型查询改写方法
- 兼容Pyserini/PyTerrier等检索后端，支持BEIR/MS MARCO基准测试
- 开源可扩展，带提示词版本管理，助力公平实验与部署
我们提出QueryGym，一个轻量级、可扩展的Python工具包，支持基于大语言模型（LLM）的查询改写。近年来，基于LLM的查询改写显著提升了检索效果，但不同研究者零散发布的实现缺乏统一框架，阻碍了公平比较、快速实验、一致基准测试与可靠部署。QueryGym通过提供统一的实现、执行与比较平台，解决该问题：(1) 提供多样LLM改写方法的Python API；(2) 与Pyserini、PyTerrier等后端兼容的检索无关接口；(3) 带版本控制和元数据追踪的集中式提示词管理系统；(4) 内置对BEIR和MS MARCO等基准的支持；(5) 完全开源且可扩展的实现，向所有研究者开放。工具包已公开于https://github.com/radinhamidi/QueryGym。
原文摘要 · Abstract (English)
We present QueryGym, a lightweight, extensible Python toolkit that supports large language model (LLM)-based query reformulation. This is an important tool development since recent work on llm-based query reformulation has shown notable increase in retrieval effectiveness. However, while different authors have sporadically shared the implementation of their methods, there is no unified toolkit that provides a consistent implementation of such methods, which hinders fair comparison, rapid experimentation, consistent benchmarking and reliable deployment. QueryGym addresses this gap by providing a unified framework for implementing, executing, and comparing llm-based reformulation methods. The toolkit offers: (1) a Python API for applying diverse LLM-based methods, (2) a retrieval-agnostic interface supporting integration with backends such as Pyserini and PyTerrier, (3) a centralized prompt management system with versioning and metadata tracking, (4) built-in support for benchmarks like BEIR and MS MARCO, and (5) a completely open-source extensible implementation available to all researchers. QueryGym is publicly available at https://github.com/radinhamidi/QueryGym.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。