arXiv:2509.19322cs.CLcs.AI2025-09

让大模型动态调用数据源元信息,减少幻觉,提升回答准确性。

Readme_AI: Dynamic Context Construction for Large Language Models

  • 通过元数据文件动态构建模型上下文,由数据提供者自定义。
  • 在NIST的Hedgehog库测试中,模型准确生成代码并避免幻觉。
  • 适合数据提供方希望提升模型对特定数据理解能力的场景。

尽管大型语言模型(LLMs)经过大量数据训练,但在面对用户特定查询时仍可能提供不准确或不可靠的信息。为提升响应实用性,本文提出一种可动态构建数据源上下文的规范。数据源所有者创建包含元数据的文件,供LLM在处理相关查询时使用。为验证该规范,我们开发了原型系统Readme_AI Model Context Protocol(MCP)服务器,从数据源获取元数据并动态构建上下文。该规范支持扩展类型,包括爬取网页、从数据仓库获取数据、下载解析论文及通用文本。上下文通过用户指定标签进行格式化和分组,帮助LLM清晰理解内容。我们在NIST开发的Hedgehog库上测试该系统,传统LLM常出现错误且无关的回答,含幻觉;而使用Readme_AI后,模型能准确推理该库及其用途,并基于提供的示例生成代码。主要贡献是提出一种可扩展协议,使LLM能动态依托专用数据提供者的内容,增强响应质量,减少幻觉。代码已开源:https://github.com/usnistgov/readme_ai。

原文摘要 · Abstract (English)

Despite being trained on significant amounts of data, Large Language Models (LLMs) can provide inaccurate or unreliable information in the context of a user's specific query. Given query-specific context significantly improves the usefulness of its responses. In this paper, we present a specification that can be used to dynamically build context for data sources. The data source owner creates the file containing metadata for LLMs to use when reasoning about dataset-related queries. To demonstrate our proposed specification, we created a prototype Readme_AI Model Context Protocol (MCP) server that retrieves the metadata from the data source and uses it to dynamically build context. Some features that make this specification dynamic are the extensible types that represent crawling web-pages, fetching data from data repositories, downloading and parsing publications, and general text. The context is formatted and grouped using user-specified tags that provide clear contextual information for the LLM to reason about the content. We demonstrate the capabilities of this early prototype by asking the LLM about the NIST-developed Hedgehog library, for which common LLMs often provides inaccurate and irrelevant responses containing hallucinations. With Readme_AI, the LLM receives enough context that it is now able to reason about the library and its use, and even generate code interpolated from examples that were included in the Readme_AI file provided by Hedgehog's developer. Our primary contribution is a extensible protocol for dynamically grounding LLMs in specialized, owner-provided data, enhancing responses from LLMs and reducing hallucinations. The source code for the Readme_AI tool is posted here: https://github.com/usnistgov/readme_ai .

大模型上下文构建元数据减少幻觉

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。