统一提取式内容选择任务,用指令驱动模型提升泛化能力。
A Unifying Scheme for Extractive Content Selection Tasks
- 用指令封装任务定义,让大模型统一处理各类内容选择问题。
- 构建首个覆盖多任务的基准数据集,支持跨任务评估与训练。
- 合成数据+迁移学习显著提升性能,适合研究者快速实验部署。
众多自然语言处理任务涉及从源文本中选取相关片段。尽管目标一致,这些内容选择任务长期被孤立研究,各自采用不同的建模方法、数据集和评估指标。本文提出指令引导的内容选择(IGCS)框架,将任务定义和实例需求以指令形式输入语言模型,实现统一建模。为此,我们构建了首个涵盖多样内容选择任务的统一基准 \\_igcsbench{},并创建了一个大规模通用合成数据集,证明其在跨任务迁移中可显著提升性能,即使缺乏特定任务的标注数据也有效。此外,我们分析了基于大模型推理时的通用问题,评估了通用评价指标,并整体验证了该框架及资源对后续内容选择模型的实用价值。代码与数据见 https://github.com/shmuelamar/igcs。
原文摘要 · Abstract (English)
A broad range of NLP tasks involve selecting relevant text spans from given source texts. Despite this shared objective, such \textit{content selection} tasks have traditionally been studied in isolation, each with its own modeling approaches, datasets, and evaluation metrics. In this work, we propose \textit{instruction-guided content selection (IGCS)} as a beneficial unified framework for such settings, where the task definition and any instance-specific request are encapsulated as instructions to a language model. To promote this framework, we introduce \igcsbench{}, the first unified benchmark covering diverse content selection tasks. Further, we create a large generic synthetic dataset that can be leveraged for diverse content selection tasks, and show that transfer learning with these datasets often boosts performance, whether dedicated training for the targeted task is available or not. Finally, we address generic inference time issues that arise in LLM-based modeling of content selection, assess a generic evaluation metric, and overall propose the utility of our resources and methods for future content selection models. Models and datasets available at https://github.com/shmuelamar/igcs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。