用AI工具自动筛选医疗创新信号，减少95%人工审阅工作量。
Horizon Scans can be accelerated using novel information retrieval and artificial intelligence tools
- 开发两个开源工具SCANAR与AIDOC，自动化处理新闻数据并排序相关性。
- AIDOC在保持95%召回率下，减少约62%的人工审查工作量。
- 适合需要快速开展医疗趋势监测的政策制定者与研究团队。
医疗领域中的前瞻性扫描需及时捕捉创新早期信号，但当前面临从新闻等非结构化数据中高效检索与分析的挑战。本研究提出SCANAR和AIDOC两款开源Python工具，用于提升扫描效率。SCANAR实现新闻文章的自动化采集与处理，支持去重与无监督相关性排序；AIDOC利用神经网络计算语义相似度，对文本按相关性重新排序，优先推送高可能相关条目供人工审阅。在12个内部数据集及4个外部基准数据集上的测试表明，SCANAR显著减少了人工依赖；AIDOC在95%召回率下实现约62%的工作量节省。与现有系统比较，性能相当但受数据特征影响。小规模案例显示，集成大语言模型可加速新闻数据中相关条目的检测。结果验证了两工具在提升信息获取效率方面的潜力，有望缓解方法学限制，推动更广泛、快速的前瞻性扫描。建议进一步优化模型，并设计融合大语言模型的新工作流与验证机制。
原文摘要 · Abstract (English)
Introduction: Horizon scanning in healthcare assesses early signals of innovation, crucial for timely adoption. Current horizon scanning faces challenges in efficient information retrieval and analysis, especially from unstructured sources like news, presenting a need for innovative tools. Methodology: The study introduces SCANAR and AIDOC, open-source Python-based tools designed to improve horizon scanning. SCANAR automates the retrieval and processing of news articles, offering functionalities such as de-duplication and unsupervised relevancy ranking. AIDOC aids filtration by leveraging AI to reorder textual data based on relevancy, employing neural networks for semantic similarity, and subsequently prioritizing likely relevant entries for human review. Results: Twelve internal datasets from horizon scans and four external benchmarking datasets were used. SCANAR improved retrieval efficiency by automating processes previously dependent on manual labour. AIDOC displayed work-saving potential, achieving around 62% reduction in manual review efforts at 95% recall. Comparative analysis with benchmarking data showed AIDOC's performance was similar to existing systematic review automation tools, though performance varied depending on dataset characteristics. A smaller case-study on our news datasets shows the potential of ensembling large language models within the active-learning process for faster detection of relevant articles across news datasets. Conclusion: The validation indicates that SCANAR and AIDOC show potential to enhance horizon scanning efficiency by streamlining data retrieval and prioritisation. These tools may alleviate methodological limitations and allow broader, swifter horizon scans. Further studies are suggested to optimize these models and to design new workflows and validation processes that integrate large language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。