让AI更准地从论文中提取专业信息,还能自动纠错并接受人类审核。
STRUCTSENSE: A Task-Agnostic Agentic Framework for Structured Information Extraction with Human-In-The-Loop Evaluation and Benchmarking
- 用知识图谱+智能体自我校验,提升复杂任务的提取能力。
- 在神经科学等三类任务中准确率达58%~100%,额外发现上千条新实体。
- 适合需要高精度、可解释信息抽取的研究者使用。
从科学文献中提取结构化信息对加速发现至关重要,但大语言模型在需专业知识的领域表现不佳且任务泛化能力差。我们提出 extsc{StructSense},一个模块化、任务无关、开源的框架,融合本体引导的符号知识、智能体自评估优化及人机协同验证,实现鲁棒的领域感知提取。在三个语义复杂度递增的任务上评估:基于模式的评估工具提取(准确率91–100%)、科学论文元数据与资源提取(整体准确率86–93%)、神经科学文献命名实体识别(8,882个实体,标签准确率58–75%)。在两个生物医学命名实体识别基准(NCBI Disease 和 S800 Species)上,系统实现≥90%宽松召回率和62.5–85.8%严格召回率,同时挖掘出1,000–3,600条超出标注的实体。本地概念映射服务在严格匹配下命中率Hits@1为62–82%,语义匹配下达68–86%。结果表明, extsc{StructSense} 在跨任务中具有强泛化能力,同时保持来源可追溯与透明性。
原文摘要 · Abstract (English)
Extracting structured information from scientific literature is critical for accelerating discovery, yet Large Language Models (LLMs) often struggle in specialized domains that require expert knowledge and generalize poorly across tasks. We introduce \textsc{StructSense}, a modular, task-agnostic, open-source framework that integrates ontology-guided symbolic knowledge, agentic self-evaluative refinement, and human-in-the-loop validation for robust domain-aware extraction. We evaluate \textsc{StructSense} on three tasks of increasing semantic complexity: schema-based extraction of assessment instruments (91--100\% accuracy), metadata and resource extraction from scientific papers (86--93\% overall), and named entity recognition (NER) from neuroscience literature (58--75\% label accuracy across 8,882 entities). On two biomedical NER benchmarks (NCBI Disease and S800 Species), the system achieves $\geq$90\% relaxed recall and 62.5--85.8\% strict recall while extracting 1,000--3,600 additional entities beyond gold annotations. The local concept mapping service achieves Hits@1 of 62--82\% under strict matching and 68--86\% under semantic matching. These results across three domains demonstrate that \textsc{StructSense} generalizes across tasks while maintaining source grounding and provenance transparency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。