小模型靠领域适配提效,大模型反而可能被干扰。
Is Domain Adaptation Always Helpful? A Frozen-Backbone Study of Cross-Domain Sentiment Transfer

- 用轻量MLP适配器在消费评论上做领域对齐。
- 金融数据集上小模型性能显著提升,电影评论无明显改善。
- 对抗性对齐会破坏专业模型原有结构,对比学习更稳定。
冻结预训练语言模型(PLM)主干的情感分析已成为常见范式,但显式领域适配的实际收益尚不明确,尤其当主干模型已具备不同程度目标领域知识时。我们通过一个受控实验,评估了Qwen3-Embedding 0.6B、4B、8B,以及RoBERTa-base和FinBERT五种冻结嵌入主干。在消费者评论上使用域对抗神经网络(DANN)、最大均值差异(MMD)和有监督对比学习(SCL)训练轻量级MLP适配器,并评估其在电影评论(SST-2)和金融新闻子集(Financial PhraseBank)上的迁移效果。结果显示:在SST-2上,无论主干规模如何,领域适配均无显著增益;而在金融子集上,通用小模型通过适配性能大幅提升。值得注意的是,对抗性对齐(DANN)会降低领域专业化模型(如FinBERT)的表现,可能破坏已有领域结构;而有监督对比损失则有助于保持该结构。初步表明,显式领域适配的有效性高度依赖于冻结主干是否已具备目标领域覆盖。
原文摘要 · Abstract (English)
Sentiment analysis with frozen pre-trained language model (PLM) backbones has become a common paradigm, yet the practical benefit of explicit domain adaptation remains unclear, particularly when backbones encode varying degrees of target-domain knowledge. We present a preliminary case study evaluating a controlled family of frozen embedding backbones (Qwen3-Embedding 0.6B, 4B, 8B), alongside RoBERTa-base and FinBERT. We train a lightweight MLP adapter on consumer reviews using Domain-Adversarial Neural Networks (DANN), Maximum Mean Discrepancy (MMD), and Supervised Contrastive Learning (SCL), and evaluate transfer to movie reviews (SST-2) and a heavily restricted subset of financial news (Financial PhraseBank). Within this constrained sample, we observe two distinct transfer patterns. On SST-2, domain adaptation provides negligible gain regardless of scale. On the financial subset, explicit domain adaptation appears to recover substantial performance for small general-purpose backbones. Notably, we find that adversarial alignment (DANN) is associated with degraded performance for domain-specialized backbones like FinBERT, consistent with erosion of pre-existing domain-specific structure, whereas supervised contrastive loss appears to preserve it. These preliminary findings suggest that the efficacy of explicit domain adaptation is highly contingent on whether the frozen backbone already possesses target-domain coverage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。