用代理失败反向驱动企业知识库构建,只保留必需内容。
Demand-Driven Context: A Methodology for Building Enterprise Knowledge Bases Through Agent Failure
- 以代理任务失败为信号,动态识别并收集缺失的知识
- 9轮测试生成46个实体的知识库,满足SRE故障管理需求
- 适合需要快速构建高精度企业知识库的团队
大语言模型代理虽具备专家级推理能力,但在企业特定任务中因缺乏领域知识(如术语、流程、系统依赖和机构决策)而持续失败——这些知识多为口耳相传的隐性知识。现有方法分为自上而下知识工程(预先构建知识库)和自下而上自动化(从任务经验学习),前者导致知识库臃肿且未经验证,后者无法获取仅存在于人类大脑中的知识。本文提出需求驱动上下文(Demand-Driven Context, DDC),一种问题导向的方法论:以代理失败为首要信号,反向确定需补充的知识。受测试驱动开发启发,DDC不预设知识,而是先给代理真实问题,让其暴露所需上下文,再仅针对必要内容进行最小化知识采集。我们定义了实体元模型,并提出收敛假设:20-30个问题循环可生成足够支持某一领域角色的知识库。在零售订单履约场景中,通过9轮针对SRE故障管理代理的测试,构建出包含46个实体的可复用知识库。最后,我们提出了支持企业规模化应用的半自动化采集与人工治理架构。
原文摘要 · Abstract (English)
Large language model agents demonstrate expert-level reasoning, yet consistently fail on enterprise-specific tasks due to missing domain knowledge -- terminology, operational procedures, system interdependencies, and institutional decisions that exist largely as tribal knowledge. Current approaches fall into two categories: top-down knowledge engineering, which documents domain knowledge before agents use it, and bottom-up automation, where agents learn from task experience. Both have fundamental limitations: top-down efforts produce bloated, untested knowledge bases; bottom-up approaches cannot acquire knowledge that exists only in human heads. We present Demand-Driven Context (DDC), a problem-first methodology that uses agent failure as the primary signal for what domain knowledge to curate. Inspired by Test-Driven Development, DDC inverts knowledge engineering: instead of curating knowledge and hoping it is useful, DDC gives agents real problems, lets them demand the context they need, and curates only the minimum knowledge required to succeed. We describe the methodology, its entity meta-model, and a convergence hypothesis suggesting that 20-30 problem cycles produce a knowledge base sufficient for a given domain role. We demonstrate DDC through a worked example in retail order fulfillment, where nine cycles targeting an SRE incident management agent produce a reusable knowledge base of 46 entities. Finally, we propose a scaling architecture for enterprise adoption with semi-automated curation and human governance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。