用大模型精准识别论文对可持续发展目标的真实贡献
Agentic AI for Improving Precision in Identifying Contributions to Sustainable Development Goals
- 用自回归大模型分析论文语义，区分真实相关与关键词巧合
- 小规模本地模型即可有效提升检索准确率，降低误检
- 适合科研机构评估可持续发展研究绩效
随着研究机构日益支持联合国可持续发展目标（SDGs），准确评估其研究成果的贡献变得愈发重要。现有方法主要依赖关键词的布尔搜索，常将偶然出现关键词的文献误判为相关成果，导致检索精度下降，影响基准比较。本研究探索使用自回归大语言模型（LLMs）作为评估代理，识别学术出版物中对具体SDG目标的真实贡献。基于通过特定关键词查询获取的学术摘要数据集，结果表明：小型、可本地部署的LLM能够有效区分与SDG目标在语义上相关的研究贡献与仅因关键词巧合被召回的文献，克服了传统方法的局限。该方法利用大模型的上下文理解能力，提供了一种可扩展的框架，有助于提升与SDG相关的研究指标，并支持机构报告。
原文摘要 · Abstract (English)
As research institutions increasingly commit to supporting the United Nations' Sustainable Development Goals (SDGs), there is a pressing need to accurately assess their research output against these goals. Current approaches, primarily reliant on keyword-based Boolean search queries, conflate incidental keyword matches with genuine contributions, reducing retrieval precision and complicating benchmarking efforts. This study investigates the application of autoregressive Large Language Models (LLMs) as evaluation agents to identify relevant scholarly contributions to SDG targets in scholarly publications. Using a dataset of academic abstracts retrieved via SDG-specific keyword queries, we demonstrate that small, locally-hosted LLMs can differentiate semantically relevant contributions to SDG targets from documents retrieved due to incidental keyword matches, addressing the limitations of traditional methods. By leveraging the contextual understanding of LLMs, this approach provides a scalable framework for improving SDG-related research metrics and informing institutional reporting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。