让大模型学会追踪文本随时间变化，尤其擅长处理稀有事件和数据少的情况。
LiFT: How to Enable In-Context Learning for Longitudinal Modelling
- 用统一指令模板+时间依赖建模，让大模型具备纵向任务的上下文学习能力
- 在零、一、三样本下均超越现有方法，对少数类提升更显著
- 适合心理健康监测、立场演化等需长期跟踪的任务
纵向自然语言处理任务如心理健康监测和立场演变，需要建模时序文本以追踪持续性并检测变化。这类任务常面临数据稀缺问题，包括罕见事件和稀疏标注。大语言模型可通过上下文学习（ICL）从少量数据中学习，在低资源场景下尤为重要。然而，面对演化交互或罕见事件识别，现有模型在零样本和少样本ICL上表现不佳。为此，我们提出LiFT——一种与模型无关的纵向指令微调框架，可引导大模型在纵向任务中实现有效的上下文学习。LiFT通过共享指令模板，融合序列依赖、课程学习和时间条件建模，统一多种任务。我们在五个纵向数据集上评估了零、一、三样本设置下的性能，结果表明：在相同ICL条件下，LiFT优于现有IFT模型。此外，LiFT对少数类带来更大提升，能有效关注最近的示例历史，并具备超越训练任务的可迁移纵向建模能力。
原文摘要 · Abstract (English)
Longitudinal NLP tasks such as mental health monitoring and stance evolution require modeling temporally ordered text to track persistence and detect change. Such tasks also suffer from data scarcity, often involving rare events and sparsely annotated data. Large Language Models (LLMs) can learn from small amounts of data through in-context learning (ICL), which is particularly important in low-data resource scenarios. However, when it comes to tracking evolving interactions or identifying rare events or changes, LLMs fall short in both zero- and few-shot ICL. To address this limitation, we introduce LiFT, a model-agnostic longitudinal Instruction Fine-Tuning framework that is able to induce ICL behaviour in LLMs for longitudinal tasks. LiFT unifies diverse tasks through a shared instruction schema that leverages sequential dependencies, curriculum learning, and temporal conditioning. We evaluate LiFT on five longitudinal datasets under zero-, one-, and three-shot settings, demonstrating that LiFT delivers superior performance to IFT models under identical ICL conditions. Furthermore, we demonstrate that LiFT yields larger gains for minority classes, meaningfully attends to recent demonstration history, and develops transferable longitudinal modeling capabilities beyond its training tasks
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。