用知识图谱增强大模型，让AI政策合规推理更准更快。
Knowledge Graph Representations for LLM-Based Policy Compliance Reasoning
- 从政策文档构建知识图谱，支持多类型推理
- 图谱增强使5个大模型在42个任务上得分提升
- 大模型自发现的开放模式效果不输正式标准
随着AI功能快速融入软件应用，其带来的风险日益增加，相关安全与合规规范也相继出台。本文提出一种智能体框架，通过从三份与AI风险相关的政策文件中构建知识图谱（KG），并利用图谱检索信息以回答政策相关问题。我们在两种本体模式下构建了知识图谱，并在涵盖六种推理类型（从实体查找至跨政策推理）的42个政策问答任务上，评估了五个大语言模型（LLMs）。采用启发式评分和大模型作为评判者双重验证。结果显示，知识图谱增强使所有五种模型性能提升，且由大模型自主发现的开放本体模式在多数任务上表现达到或超过正式本体标准。
原文摘要 · Abstract (English)
The risks posed by AI features are increasing as they are rapidly integrated into software applications. In response, regulations and standards for safe and secure AI have been proposed. In this paper, we present an agentic framework that constructs knowledge graphs (KGs) from AI policy documents and retrieves policy-relevant information to answer questions. We build KGs from three AI risk-related polices under two ontology schemas, and then evaluate five LLMs on 42 policy QA tasks spanning six reasoning types, from entity lookup to cross-policy inference, using both heuristic scoring and an LLM-as-judge. KG augmentation improves scores for all five models, and an open, LLM-discovered schema matches or exceeds the formal ontology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。