用本体引导+去重感知,从杂乱文档中精准构建知识图谱。
An Ontology-Guided, Deduplication-Aware Extraction Layer for Knowledge Graph Construction from Heterogeneous Documents
- 基于本体动态注入提示,减少94%的领域词表开销。
- 在情报语料上检索召回率从70%提升至95%,无误合并。
- 六种无需模型推理的去重算法,解决七类隐蔽质量缺陷。
大语言模型虽能流畅提取实体与关系,但存在类型词汇分散、同人异名、关系重复及同名不同人隐性混淆等问题。本文设计并实证优化了一个生产级抽取层,将实时文档流转化为符合形式化本体的验证知识图谱。系统通过Kafka接收文档元数据,按格式分发至对应处理模块,利用本地部署的微调版Qwen3.5-9B模型分两轮抽取实体与关系。核心在于本体引导抽取:通过嵌入相似度实时从图数据库检索相关本体片段注入提示,较静态领域切片降低约94%的词表开销。抽取结果经五阶段精炼:确定性清洗、跨块合并、关系二次抽取、六种无需模型推理的去重算法,以及冲突保护的嵌入解析子系统。在情报语料上的评估显示,搜索召回率由约70%提升至95%,无误合并,修复了七类隐性质量问题,涵盖单字符截断错误及带头衔前缀实体的系统性重复。
原文摘要 · Abstract (English)
Large language models extract entities and relationships from unstructured documents fluently but inconsistently: type vocabularies fracture across documents, the same person surfaces under several name variants, relationships duplicate, and distinct individuals who share a name risk silent conflation. This paper presents the design, implementation, and empirical refinement of a production extraction layer that converts a live document stream into a validated knowledge graph aligned to a formal ontology. The system consumes document metadata from Kafka, routes PDF, spreadsheet, Office, and image content through handlers built for each format, and extracts entities and relationships in two passes using a locally hosted Qwen3.5-9B model tuned on the ontology. Its distinguishing component is ontology-guided extraction: the relevant slice of a curated ontology is retrieved live from a graph database by embedding similarity and injected into the extraction prompt, reducing catalog overhead by about 94 percent relative to static domain slices. Extracted results then pass through a refinement pipeline of five stages: deterministic cleaning, merging across chunks, a second pass for relationships, six deduplication algorithms that require no model inference, and an embedding resolution subsystem whose conflict guard no similarity score can override. Evaluation on intelligence corpora improved search recall from roughly 70 to 95 percent with no false merges, and corrected seven classes of silent quality defect, ranging from a bug that truncated source text by a single character to the systematic duplication of entities that carried title prefixes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。