arXiv:2506.08422cs.AI2025-06被引 3

用大模型+专家校准,自动构建领域知识体系

Transforming Expert Knowledge into Scalable Ontology via Large Language Models

  • 结合大模型与专家反馈,迭代优化提示词生成映射关系
  • 在概念重要性映射任务上达F1 0.97,超人工基准0.68
  • 适合需要高质量、可解释知识体系的科研与工业场景

统一连贯的分类体系对特定领域知识表示至关重要,因术语多样需映射到核心概念。传统人工对齐依赖专家评审概念对,但成本高昂且易因主观判断产生分歧。现有自动化方法虽有进展,但在处理细微语义关系和跨领域一致性方面仍受限,常无法应对上下文相关的概念映射,且缺乏透明推理过程。本文提出一种新框架,将大语言模型(LLMs)与专家校准、迭代提示优化相结合,实现自动化分类体系对齐。该方法融合专家标注样本、多阶段提示工程及人工验证,引导LLM生成分类链接及其支持理由。在特定领域概念重要性映射任务中,本框架取得F1分数0.97,显著高于人类基准0.68。结果表明,该方法可在保持高质量映射的同时实现分类体系对齐的规模化,并通过专家监督保障模糊案例的准确性。

原文摘要 · Abstract (English)

Having a unified, coherent taxonomy is essential for effective knowledge representation in domain-specific applications as diverse terminologies need to be mapped to underlying concepts. Traditional manual approaches to taxonomy alignment rely on expert review of concept pairs, but this becomes prohibitively expensive and time-consuming at scale, while subjective interpretations often lead to expert disagreements. Existing automated methods for taxonomy alignment have shown promise but face limitations in handling nuanced semantic relationships and maintaining consistency across different domains. These approaches often struggle with context-dependent concept mappings and lack transparent reasoning processes. We propose a novel framework that combines large language models (LLMs) with expert calibration and iterative prompt optimization to automate taxonomy alignment. Our method integrates expert-labeled examples, multi-stage prompt engineering, and human validation to guide LLMs in generating both taxonomy linkages and supporting rationales. In evaluating our framework on a domain-specific mapping task of concept essentiality, we achieved an F1-score of 0.97, substantially exceeding the human benchmark of 0.68. These results demonstrate the effectiveness of our approach in scaling taxonomy alignment while maintaining high-quality mappings and preserving expert oversight for ambiguous cases.

知识图谱大模型应用专家系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。