arXiv:2507.00601cs.CL2025-07被引 15

轻量适配+对齐提示,让大模型在低资源语言下快速稳定迁移。

Transferable Modeling Strategies for Low-Resource LLM Tasks: A Prompt and Alignment-Based Approach

  • 用软提示与对齐损失引导模型吸收目标语言结构特征。
  • 在极低数据下跨语言任务准确率超现有方法,稳定性更优。
  • 适合需要快速适配新语言/任务的工业级多语应用。

本文针对大语言模型在低资源语言场景下迁移与适配能力有限的问题,提出一种融合知识迁移模块与参数高效微调策略的统一框架。该方法引入知识对齐损失与软提示调优,指导模型在极少标注数据下有效吸收目标语言或任务的结构特征,提升泛化性能与训练稳定性。框架包含轻量适配模块以降低计算开销,并通过冻结策略与提示注入,在保留原始知识的同时实现新任务的快速适应。研究还开展稳定性分析与合成伪数据迁移实验,系统评估方法在不同低资源任务中的适用性与鲁棒性。实验结果表明,相较于现有的多语言预训练模型及主流迁移方法,该方法在跨语言任务(如MLQA、XQuAD、PAWS-X)上表现更优,尤其在极端数据稀缺条件下优势显著。所提方法具备强通用性与可扩展性,既增强任务特定适配能力,又保留大模型的通用能力,适用于复杂语义建模与多语言处理任务。

原文摘要 · Abstract (English)

This paper addresses the limited transfer and adaptation capabilities of large language models in low-resource language scenarios. It proposes a unified framework that combines a knowledge transfer module with parameter-efficient fine-tuning strategies. The method introduces knowledge alignment loss and soft prompt tuning to guide the model in effectively absorbing the structural features of target languages or tasks under minimal annotation. This enhances both generalization performance and training stability. The framework includes lightweight adaptation modules to reduce computational costs. During training, it integrates freezing strategies and prompt injection to preserve the model's original knowledge while enabling quick adaptation to new tasks. The study also conducts stability analysis experiments and synthetic pseudo-data transfer experiments to systematically evaluate the method's applicability and robustness across different low-resource tasks. Experimental results show that compared with existing multilingual pre-trained models and mainstream transfer methods, the proposed approach achieves higher performance and stability on cross-lingual tasks such as MLQA, XQuAD, and PAWS-X. It demonstrates particularly strong advantages under extremely data-scarce conditions. The proposed method offers strong generality and scalability. It enhances task-specific adaptability while preserving the general capabilities of large language models. This makes it well-suited for complex semantic modeling and multilingual processing tasks.

低资源提示调优多语言轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。