无需更新权重，用无监督方法修正语言模型的时间分布错位。
TARDIS: Mitigating Temporal Misalignment via Representation Steering
- 从无标签数据提取调节向量，动态调整模型表示
- 不需微调即可提升下游任务性能，效果优于基准方法
- 适合无法获取目标时期数据或时间信息的场景
语言模型常受数据时间分布变化导致的性能退化困扰。持续更新模型成本高昂。能否在不更新模型权重的情况下实现适应？我们提出 TARDIS，一种无监督表示编辑方法。TARDIS 从无标签数据中提取调节向量，调整模型表示以更好地匹配目标时间段的数据分布。实验表明，TARDIS 在无需微调的情况下提升下游任务性能，即使在无法获得确切目标时期数据时仍可缓解时间错位问题，且在推理时未知目标数据时间信息的情况下依然高效。
原文摘要 · Abstract (English)
Language models often struggle with temporal misalignment, performance degradation caused by shifts in the temporal distribution of data. Continuously updating models to avoid degradation is expensive. Can models be adapted without updating model weights? We present TARDIS, an unsupervised representation editing method that addresses this challenge. TARDIS extracts steering vectors from unlabeled data and adjusts the model's representations to better align with the target time period's distribution. Our experiments reveal that TARDIS enhances downstream task performance without the need for fine-tuning, can mitigate temporal misalignment even when exact target time period data is unavailable, and remains efficient even when the temporal information of the target data points is unknown at inference time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。