arXiv:2606.18875cs.CL2026-06

用合成数据蒸馏,让小模型在金融情感分析中表现超越大模型。

Efficient Financial Language Understanding via Distillation with Synthetic Data

论文配图:Efficient Financial Language Understanding via Distillation with Synthetic Data
图 1 · 摘自论文原文
  • 通过聚类选种子生成合成数据,提升小模型训练效率。
  • 仅需少量人工标注,小模型在复杂文本上超越教师模型。
  • 适合金融领域数据少、标注成本高的场景使用。

大型指令跟随模型虽强大但部署成本高,尤其在金融领域受限于数据保密和专家标注成本。本文提出一种基于合成数据蒸馏的高效金融情感分析框架,将大模型知识迁移到小型学生模型。该框架适用于低资源场景:先人工收集少量真实样本并标注,再通过聚类选择种子,利用结构化少样本提示生成合成数据。实验表明,基于聚类的种子选择比随机采样生成更具代表性的合成数据,使紧凑模型在极小监督下仍表现优异。值得注意的是,在更复杂嘈杂的文本领域,仅用合成数据训练的小模型甚至优于教师模型,且在正式文本上保持竞争力。该框架为金融NLP中的资源高效领域适配提供了可行路径,极大减少人工标注投入。

原文摘要 · Abstract (English)

Large instruction-following models are powerful but costly to deploy, particularly in finance, where labelled data are limited by confidentiality and expert annotation cost. We present an efficient framework for financial sentiment analysis through distillation with synthetic data, transferring knowledge from a large instruction-tuned teacher to compact student models. The framework is designed for low-resource conditions, where a small set of real examples are collected and labelled by hand. The framework then clusters the examples and uses the clusters to select seeds for generating synthetic examples via structured few-shot prompting. Experiments show that clustering-based seed selection yields more representative synthetic data than random sampling, enabling compact models to achieve strong performance with minimal supervision. Notably, on a more complex and noisy text domain, the compact model trained on the complete synthetic-seed corpus even outperforms the teacher model, while remaining competitive on formal text. The framework provides a practical route toward resource-efficient domain adaptation in financial NLP with minimal human labelling effort.

金融NLP知识蒸馏合成数据小样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。