让大模型学会修改自由文本中的常识知识，更真实可靠。
Commonsense Knowledge Editing Based on Free-Text in LLMs
- 提出新方法定位自由文本中的常识知识分布
- 动态感知模块精准找到需修改的参数位置
- 适用于日常对话、写作等场景中的常识更新
知识编辑技术对维持大语言模型（LLMs）的准确性和时效性至关重要。然而，现有方法忽视了现实世界中以自由文本形式存在的大量常识知识，这类知识具有范围广、内容长、非实例化等特点。以往方法（如MEMIT）仅针对单个词或实体进行编辑，不适用于自由文本形式的常识知识。为此，本文从知识定位和知识编辑两方面开展实验：首先提出自由文本知识定位方法（KLFT），揭示常识知识在MLP与注意力层中的分布挑战及分散特性；随后提出动态感知编辑方法（DEM），利用动态感知模块定位对应常识知识的参数位置，并通过知识编辑模块完成更新。DEM充分挖掘了MLP与注意力层潜力，成功实现了基于自由文本的常识知识编辑。实验结果表明，DEM具备优异的编辑性能。
原文摘要 · Abstract (English)
Knowledge editing technology is crucial for maintaining the accuracy and timeliness of large language models (LLMs) . However, the setting of this task overlooks a significant portion of commonsense knowledge based on free-text in the real world, characterized by broad knowledge scope, long content and non instantiation. The editing objects of previous methods (e.g., MEMIT) were single token or entity, which were not suitable for commonsense knowledge in free-text form. To address the aforementioned challenges, we conducted experiments from two perspectives: knowledge localization and knowledge editing. Firstly, we introduced Knowledge Localization for Free-Text(KLFT) method, revealing the challenges associated with the distribution of commonsense knowledge in MLP and Attention layers, as well as in decentralized distribution. Next, we propose a Dynamics-aware Editing Method(DEM), which utilizes a Dynamics-aware Module to locate the parameter positions corresponding to commonsense knowledge, and uses Knowledge Editing Module to update knowledge. The DEM method fully explores the potential of the MLP and Attention layers, and successfully edits commonsense knowledge based on free-text. The experimental results indicate that the DEM can achieve excellent editing performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。