让大模型主动找知识,提升跨领域迁移效率。
LEKA:LLM-Enhanced Knowledge Augmentation
- 大模型自主检索外部知识库,匹配目标领域的关键信息。
- 在特征空间和概率分布上对齐知识,实现高效融合。
- 显著降低计算成本,适合需要知识迁移的AI应用。
人类擅长类比学习与知识迁移,尤其能精准判断知识来源的适用性。从模型视角看,若能自主检索用于迁移或决策的有用知识,便能从被动获取转向主动学习。虽然向模型注入知识较简单(只需更多训练和知识库),但教会模型识别可迁移的知识更具挑战。为此,我们提出LEKA方法,实现主动知识增强:从目标领域文本中提取关键信息,从外部数据源检索相关数据,并在特征空间与边缘概率分布上对齐。大量实验证明,该方法在多个领域均优于传统方法,在降低计算开销、自动化数据对齐和优化迁移效果方面表现突出。
原文摘要 · Abstract (English)
Humans excel in analogical learning and knowledge transfer and, more importantly, possess a unique understanding of identifying appropriate sources of knowledge. From a model's perspective, this presents an interesting challenge. If models could autonomously retrieve knowledge useful for transfer or decision-making to solve problems, they would transition from passively acquiring to actively accessing and learning from knowledge. However, filling models with knowledge is relatively straightforward -- it simply requires more training and accessible knowledge bases. The more complex task is teaching models about which knowledge can be analogized and transferred. Therefore, we design a knowledge augmentation method, LEKA, for knowledge transfer that actively searches for suitable knowledge sources that can enrich the target domain's knowledge. This LEKA method extracts key information from the target domain's textual information, retrieves pertinent data from external data libraries, and harmonizes retrieved data with the target domain data in feature space and marginal probability measures. We validate the effectiveness of our approach through extensive experiments across various domains and demonstrate significant improvements over traditional methods in reducing computational costs, automating data alignment, and optimizing transfer learning outcomes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。