提出高效可落地的逻辑推理框架,让大模型生成内容更可信。
Enhanced and Efficient Reasoning in Large Learning Models
- 将文本数据转为显式关系编码,强化对象间关联表达
- 使核心关系规则学习变为多项式时间可解,提升推理效率
- 兼容现有软硬件,适合语言、视觉等多模态应用
当前大语言模型虽能生成流畅文本,但其内容可信度缺乏原理支撑。传统增强推理方法计算成本过高,难以实用。本文提出一种高效且具原则性的推理方法:第一阶段将输入数据重编码为更显式表达对象间关系的Unary Relational Integracode;第二阶段采用标准或简化机器学习流程,同时学习预测这些关系。该方法可视为构建世界模型,适用于自然语言、视觉与动作等多领域,将同一对象的多属性集中表示,而非分散于不同引用中。我们以鲁棒逻辑(Robust Logic)系统验证其优势,证明该编码虽简洁,却使世界中核心关系规则的学习在特定意义下变为多项式时间可解,其复杂度取决于规则本身。这支持了单次调用内及多次调用间的可靠推理。
原文摘要 · Abstract (English)
In current Large Language Models we can trust the production of smoothly flowing prose on the basis of the principles of machine learning. However, there is no comparably principled basis to justify trust in the content of the text produced. It appears to be conventional wisdom that addressing this issue by adding more principled reasoning is not computationally affordable. Here we propose a principled method of reasoning that is efficient enough to be practical for large language models. Further, the method allows the retention of much of the currently used software and hardware base. Our method for improving the functioning of large language models consists of a first stage of preprocessing that recodes the data to a Unary Relational Integracode that is more explicit about the relationships among the objects described in the text, followed as a second stage by a standard but possibly streamlined machine learning process that then also learns to predict these relationships. The method may be viewed as realizing a world model and applying beyond natural language, to vision and actions, for example, where the multiple properties of an object referred to in an input are brought together explicitly, rather than remaining distributed in the various references to it in the input. We articulate its advantages in terms of Robust Logic, a system for performing principled chaining on learned, and hence uncertain, information. We show that this recoding has the surprising and fortuitous property that, while succinct, it makes the task of learning a core subset of relational rules that hold in the world described in the training data polynomial time learnable in a defined sense, the polynomial depending on the complexity of the rule. This gives support for sound reasoning within each single call of the learned classifier as well as between multiple calls.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。