让信息检索模拟更易用:可视化拖拽设计实验流程
IIRSim Studio: A Dashboard for User Simulation

- 用图形界面拖拽构建模拟流程,降低使用门槛
- 支持组件版本管理与共享,提升实验可复现性
- 适合新手学习和专家开展大规模模拟实验
用户模拟是信息检索领域一种有价值的评估方法,可实现低成本实验和反事实分析。然而现有模拟框架多为代码驱动,需大量配置工作,限制了其普及并影响实验可复现性。问题不在于模拟引擎本身,而在于缺乏将实验设计、执行与共享整合为可验证工作流的基础设施。本文提出 IIRSim Studio,一个基于网页的集成工作台,包含四项贡献:(1)在模拟框架基础上提供可视化环境,用于构建模拟流程,既适合初学者学习概念,也支持专家开展大规模实验;(2)基于 Git 的组件生命周期管理,支持自定义组件的编写、版本控制与运行时注入;(3)基于实验包与环境模板的溯源模型,明确复现范围;(4)通过重部署 Sim4IA 微任务,演示共享任务工作流。IIRSim Studio 可作为托管服务使用,也可容器化本地部署。
原文摘要 · Abstract (English)
User simulation is a valuable methodology for evaluation in Information Retrieval (IR), enabling low-cost experimentation and counterfactual analysis. However, existing simulation frameworks are primarily code-centric libraries that require substantial setup effort, which limits adoption and hinders reproducibility. The bottleneck is not the simulation engines themselves, but the lack of infrastructure connecting experiment design, execution, and sharing into a single verifiable workflow. This paper introduces IIRSim Studio, a web-based workbench that addresses this gap through four contributions: (1) a visual environment for composing simulation pipelines on top of simulation frameworks, serving both novices learning simulation concepts and experts piloting large-scale experiments; (2) a component lifecycle that supports authoring, versioning, and sharing custom simulation components through Git-backed storage and runtime injection; (3) a provenance model based on experiment bundles and environment templates that makes the scope of replication explicit; and (4) a shared-task workflow, demonstrated through the re-deployment of a Sim4IA micro-task. IIRSim Studio is available as a hosted service and as a portable containerized deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。