自动构建AI领域论文排行榜，帮研究者快速追踪最新进展。
League: Leaderboard Generation on Demand
- 分四步自动化收集论文、提取实验数据、生成榜单、评估质量
- 在多个数据集上生成的榜单与人工整理结果高度一致
- 适合追踪前沿进展的研究者和项目负责人
本文提出一种名为LAG（Leaderboard Auto Generation）的框架，用于在人工智能等快速演进领域中自动构建特定研究主题的排行榜。面对每日大量更新的论文，研究人员难以逐一跟踪方法、结果与实验设置，亟需高效的自动化榜单生成工具。尽管大语言模型（LLMs）在该任务中展现潜力，但多文档摘要、榜单生成及实验公平比较仍面临挑战。LAG通过系统化流程——论文收集、实验结果提取与整合、榜单生成、质量评估——解决上述问题。本工作贡献包括完整的排行榜构建方案、可靠的评估方法以及实验证明生成榜单具有高质量。
原文摘要 · Abstract (English)
This paper introduces Leaderboard Auto Generation (LAG), a novel and well-organized framework for automatic generation of leaderboards on a given research topic in rapidly evolving fields like Artificial Intelligence (AI). Faced with a large number of AI papers updated daily, it becomes difficult for researchers to track every paper's proposed methods, experimental results, and settings, prompting the need for efficient automatic leaderboard construction. While large language models (LLMs) offer promise in automating this process, challenges such as multi-document summarization, leaderboard generation, and experiment fair comparison still remain under exploration. LAG solves these challenges through a systematic approach that involves the paper collection, experiment results extraction and integration, leaderboard generation, and quality evaluation. Our contributions include a comprehensive solution to the leaderboard construction problem, a reliable evaluation method, and experimental results showing the high quality of leaderboards.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。