让大模型适应时间演变，用低维流形提升效率与效果
Manifold-Aware Temporal Domain Generalization for Large Language Models
- 在低秩适配子空间中构建时间流形，约束模型更新方向
- 相比全参数更新，计算量大幅降低且性能更优
- 适合需要长期稳定部署的大模型应用场景
大型语言模型在真实场景中面临持续的数据时间分布偏移问题。现有时间域泛化方法在全参数空间建模，对现代大模型而言计算不可行。本文提出参数高效微调下的几何重表述：模型演化背后的低维时间结构可在参数高效重参数化下保持，无需在高维参数空间操作。基于此，我们提出流形感知时间LoRA（MaT-LoRA），将时间更新限制在共享的低维流形上，并通过结构化的时间核心建模其演化。该重参数化显著降低时间建模复杂度，同时保持强表达能力。在合成及真实数据集（包括科学文献、新闻媒体、评论评分）上的大量实验表明，MaT-LoRA在实现优异时间泛化性能的同时，具备对大模型的实际可扩展性。
原文摘要 · Abstract (English)
Temporal distribution shifts are pervasive in real-world deployments of Large Language Models (LLMs), where data evolves continuously over time. While Temporal Domain Generalization (TDG) seeks to model such structured evolution, existing approaches characterize model adaptation in the full parameter space. This formulation becomes computationally infeasible for modern LLMs. This paper introduces a geometric reformulation of TDG under parameter-efficient fine-tuning. We establish that the low-dimensional temporal structure underlying model evolution can be preserved under parameter-efficient reparameterization, enabling temporal modeling without operating in the ambient parameter space. Building on this principle, we propose Manifold-aware Temporal LoRA (MaT-LoRA), which constrains temporal updates to a shared low-dimensional manifold within a low-rank adaptation subspace, and models its evolution through a structured temporal core. This reparameterization dramatically reduces temporal modeling complexity while retaining expressive power. Extensive experiments on synthetic and real-world datasets, including scientific documents, news publishers, and review ratings, demonstrate that MaT-LoRA achieves superior temporal generalization performance with practical scalability for LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。