提出知识解耦框架,让模型持续学习时更高效、泛化更强。
K-DeCore: Facilitating Knowledge Transfer in Continual Structured Knowledge Reasoning via Knowledge Decoupling
- 将推理过程拆分为任务相关与无关两阶段,减少参数膨胀。
- 在四个数据集上优于现有方法,支持多种大模型架构。
- 适合需要长期学习复杂知识推理的场景,如智能问答系统。
持续结构化知识推理(CSKR)旨在训练模型处理序列任务,每个任务需将自然语言问题转化为基于结构化知识的查询。现有通用持续学习方法在该任务中面临严重挑战:对异构结构化知识泛化能力差,且随任务增加导致参数增长、推理效率下降。为此,我们提出新型CSKR框架K-DeCore,采用固定可调参数数量。不同于以往方法,K-DeCore引入知识解耦机制,将推理过程拆分为任务特定与任务无关阶段,有效弥合跨任务差异。在此基础上,K-DeCore融合双视角记忆固化机制,分别优化两个阶段,并提出结构引导伪数据生成策略,进一步提升模型泛化能力。在四个基准数据集上的大量实验表明,相较于现有持续学习方法,K-DeCore在多个指标上表现更优,适用于多种骨干大语言模型。
原文摘要 · Abstract (English)
Continual Structured Knowledge Reasoning (CSKR) focuses on training models to handle sequential tasks, where each task involves translating natural language questions into structured queries grounded in structured knowledge. Existing general continual learning approaches face significant challenges when applied to this task, including poor generalization to heterogeneous structured knowledge and inefficient reasoning due to parameter growth as tasks increase. To address these limitations, we propose a novel CSKR framework, \textsc{K-DeCore}, which operates with a fixed number of tunable parameters. Unlike prior methods, \textsc{K-DeCore} introduces a knowledge decoupling mechanism that disentangles the reasoning process into task-specific and task-agnostic stages, effectively bridging the gaps across diverse tasks. Building on this foundation, \textsc{K-DeCore} integrates a dual-perspective memory consolidation mechanism for distinct stages and introduces a structure-guided pseudo-data synthesis strategy to further enhance the model's generalization capabilities. Extensive experiments on four benchmark datasets demonstrate the superiority of \textsc{K-DeCore} over existing continual learning methods across multiple metrics, leveraging various backbone large language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。