构建动态知识编辑数据集,提升模型实时更新能力。
Aligning Language Models with Real-time Knowledge Editing
- 提出CRAFT动态数据集,涵盖时间、常识等多维度评测
- KEDAS方法在新旧数据集上均显著优于已有技术
- 适合关注模型持续学习与实时知识更新的研究者
知识编辑旨在高效修正语言模型中过时的知识,同时保持原有能力。主流知识编辑数据集多为静态,难以跟上现实世界知识的快速演变。本文提出CRAFT——一个持续演化的实时世界知识编辑数据集,涵盖时间局部性、常识局部性、复合可迁移性和别名可迁移性四项评估维度,对知识编辑提出了全面且具有挑战性的测试标准,现有方法在此基准上表现难达均衡。为实现灵活的实时知识编辑,我们提出KEDAS:一种新型知识编辑对齐范式,包含多样化的编辑增强和自适应后对齐推理机制,在CRAFT及传统数据集上均显著优于以往方法。本工作有望推动知识编辑从静态更新转向动态演化。
原文摘要 · Abstract (English)
Knowledge editing aims to modify outdated knowledge in language models efficiently while retaining their original capabilities. Mainstream datasets for knowledge editing are predominantly static and fail to keep in pace with the evolving real-world knowledge. In this work, we introduce CRAFT, an ever-evolving real-world dataset for knowledge editing. It evaluates models on temporal locality, common-sense locality, composite portability and alias portability, providing a comprehensive and challenging evaluation for knowledge editing, on which previous methods hardly achieve balanced performance. Towards flexible real-time knowledge editing, we propose KEDAS, a novel paradigm of knowledge editing alignment featuring diverse edit augmentation and self-adaptive post-alignment inference, exhibiting significant performance gain on both CRAFT and traditional datasets compared to previous methods. We hope this work may serve as a catalyst for shifting the focus of knowledge editing from static update to dynamic evolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。