用激活工程让大模型记住特定时间的事实，无需训练即可提升准确性。
Temporal Alignment of Time Sensitive Facts with Activation Engineering
- 通过调整模型激活值，将大模型对齐到特定时间点。
- 在相对和明确提示下分别提升44%和16%的准确率。
- 比微调更高效，无需数据集或额外训练，适合快速部署。
大型语言模型（LLMs）在涵盖多个领域和时间段的多样且常有冲突的知识上进行训练。部分知识仅在特定时间上下文中有效，例如回答“2022年美国总统是谁？”确保模型生成与时间相符的回答对保持相关性和准确性至关重要。本文探索了激活工程作为方法，用于对齐大模型的时间事实，无需任何训练或数据集构建。研究中采用激活工程技术，将三个版本的LLaMA 2模型锚定至特定时间点，并考察不同注入层和提示策略的影响。实验表明，在相对提示和明确提示下分别实现最高44%和16%的性能提升，达到与Zhao等人（2024）提出的微调方法相当的效果。值得注意的是，本方法在获得相近性能的同时，计算效率显著更高，且无需预先对齐的数据集。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are trained on diverse and often conflicting knowledge spanning multiple domains and time periods. Some of this knowledge is only valid within specific temporal contexts, such as answering the question, "Who is the President of the United States in 2022?" Ensuring LLMs generate time appropriate responses is crucial for maintaining relevance and accuracy. In this work we explore activation engineering as a method for temporally aligning LLMs to improve factual recall without any training or dataset creation. In this research we explore an activation engineering technique to ground three versions of LLaMA 2 to specific points in time and examine the effects of varying injection layers and prompting strategies. Our experiments demonstrate up to a 44% and 16% improvement in relative and explicit prompting respectively, achieving comparable performance to the fine-tuning method proposed by Zhao et al. (2024) . Notably, our approach achieves similar results to the fine-tuning baseline while being significantly more computationally efficient and requiring no pre-aligned datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。