自动化竞赛提交与评估系统，解决多参赛者环境兼容难题。
Online Submission and Evaluation System Design for Competition Operations
- 构建在线系统实现竞赛提交与评估全流程自动化
- 通过隔离环境评估，确保评测结果一致可靠
- 已在多个机器人路径规划竞赛中成功应用
研究社区在各领域建立了基准数据集以比较算法性能，但因论文发表分散、多数声称达到前沿水平，追踪进展困难。为此，研究社区常举办周期性竞赛以评估算法表现，从而跟踪领域进展。然而，这类竞赛带来显著运营负担：组织者需处理大量提交，且参赛者使用不同开发环境，导致评测时存在兼容性问题。本文提出一种在线竞赛系统，自动化完成竞赛的提交与评估流程。该系统使组织者能高效管理海量提交，并利用隔离环境进行评测，保障结果一致性。系统已成功应用于多个竞赛，包括网格路径规划竞赛和机器人竞跑联赛。
原文摘要 · Abstract (English)
Research communities have developed benchmark datasets across domains to compare the performance of algorithms and techniques However, tracking the progress in these research areas is not easy, as publications appear in different venues at the same time, and many of them claim to represent the state-of-the-art. To address this, research communities often organise periodic competitions to evaluate the performance of various algorithms and techniques, thereby tracking advancements in the field. However, these competitions pose a significant operational burden. The organisers must manage and evaluate a large volume of submissions. Furthermore, participants typically develop their solutions in diverse environments, leading to compatibility issues during the evaluation of their submissions. This paper presents an online competition system that automates the submission and evaluation process for a competition. The competition system allows organisers to manage large numbers of submissions efficiently, utilising isolated environments to evaluate submissions. This system has already been used successfully for several competitions, including the Grid-Based Pathfinding Competition and the League of Robot Runners competition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。