用大模型加速论文筛选，1分钟可处理100-300篇。
AISysRev -- LLM-based Tool for Title-abstract Screening
- 基于LLM实现标题摘要自动筛选，支持多模型并行调用。
- 实测每分钟处理100至300篇，显著提升筛选效率。
- 适合需要批量处理文献的系统综述研究者使用。
系统性综述耗时费力，尤其在筛选阶段面对海量论文时更为突出。近年研究表明，大语言模型（LLMs）可用于标题与摘要的初步筛选，辅助人工决策。为此，我们开发了AISysRev，一个基于LLM的筛选工具，以容器化Web应用形式部署。该工具接收包含论文标题和摘要的CSV文件，用户可自定义纳入与排除标准。支持多种LLM，如Gemini、Claude、Mistral或通过OpenRouter调用ChatGPT，也兼容本地部署模型及符合OpenAI SDK的任意模型。AISysRev提供零样本与少样本提示策略，并通过界面展示模型结果，辅助人工判断。模型调用支持并行处理，实际筛查速度达每分钟100至300篇，具体取决于模型与运行环境。为验证实用性，我们以137篇论文开展定性试点研究，发现论文可分为四类：易纳入、易排除、边界纳入、边界排除。其中边界案例易出错，凸显人工干预必要性。虽然大模型无法替代人类判断，但能大幅减轻评估海量文献的工作负担。
原文摘要 · Abstract (English)
Conducting systematic reviews is laborious. In the screening or study selection phase, the number of papers can be overwhelming. Recent research has demonstrated that large language models (LLMs) can perform title-abstract screening and support humans in the task. To this end, we developed AISysRev, an LLM-based screening tool implemented as a containerized web application. The tool accepts CSV files containing paper titles and abstracts. Users specify inclusion and exclusion criteria. Multiple different LLMs can be used, such as Gemini, Claude, Mistral or ChatGPT via OpenRouter. We also support locally hosted models and any model compatible with the OpenAI SDK. AISysRev implements both zero-shot and few-shot prompting, and also allows for manual screening through interfaces that display LLM results as guidance for human reviewers. LLM calls are parallelized, meaning screening speed is typically between 100 to 300 papers per minute, depending on the model and the host. To demonstrate the tool's use in practice, we conducted a qualitative trial study with 137 papers using the tool. Our findings indicate that papers can be classified into four categories: Easy Includes, Easy Excludes, Boundary Includes, and Boundary Excludes. The Boundary cases, where LLMs are prone to errors, highlight the need for human intervention. While LLMs do not replace human judgment in systematic reviews, they can reduce the burden of assessing large volumes of scientific literature. Video: https://www.youtube.com/watch?v=HeblemlgnAQ Tool: https://github.com/EvoTestOps/AISysRev
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。