用少量标注数据高效训练轻量NLP模型，效果超GPT-4o。
LOCUS: A System and Method for Low-Cost Customization for Universal Specialization
- 通过检索+生成补全数据，仅用少量样本完成模型定制。
- 模型仅需5%内存就保持99%全量微调准确率。
- 适合资源有限但追求高性能的NLP部署场景。
我们提出LOCUS（低代价通用专业化定制系统），一种利用少量样本高效构建与训练NLP模型的流水线。该方法通过定向检索、上下文数据生成合成样本，并采用全量或低秩（LoRA）参数调整进行微调。针对命名实体识别（NER）和文本分类（TC）任务，LOCUS在多个基准上持续优于强基线（包括GPT-4o），显著降低训练成本与模型体积。其优化后模型保留99%全量微调准确率，内存占用仅为原模型的5%，且在多个任务上以不足1%的参数量超越GPT-4o。
原文摘要 · Abstract (English)
We present LOCUS (LOw-cost Customization for Universal Specialization), a pipeline that consumes few-shot data to streamline the construction and training of NLP models through targeted retrieval, synthetic data generation, and parameter-efficient tuning. With only a small number of labeled examples, LOCUS discovers pertinent data in a broad repository, synthesizes additional training samples via in-context data generation, and fine-tunes models using either full or low-rank (LoRA) parameter adaptation. Our approach targets named entity recognition (NER) and text classification (TC) benchmarks, consistently outperforming strong baselines (including GPT-4o) while substantially lowering costs and model sizes. Our resultant memory-optimized models retain 99% of fully fine-tuned accuracy while using barely 5% of the memory footprint, also beating GPT-4o on several benchmarks with less than 1% of its parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。