让大模型更懂风格而非内容，显著提升跨领域分类表现
Controlling Out-of-Domain Gaps in LLMs for Genre Classification and Generated Text Detection
- 用可控机制屏蔽主题特征，引导模型关注文本风格
- 跨领域测试下准确率提升最高达20个百分点
- 适合需要强泛化能力的文本分类与生成检测场景
本研究发现，现代大型语言模型（如GPT-4）在非主题分类任务中仍存在与以往预训练模型（如BERT）相同的域外性能差距。我们在两类任务上验证：1）文体分类；2）生成文本检测。当上下文学习（ICL）示例来自一个领域（如旅游），而测试数据来自另一领域（如历史）时，分类性能显著下降。为此，我们提出一种方法，控制分类过程中哪些预测特征被使用或排除。该方法确保主题特征被忽略，模型被引导关注风格而非内容特征。在少样本设置下，该方法使域外差距缩小最多达20个百分点。作为基线的链式思维（CoT）方法效果有限，而我们的方法始终提升域间迁移性能。
原文摘要 · Abstract (English)
This study demonstrates that the modern generation of Large Language Models (LLMs, such as GPT-4) suffers from the same out-of-domain (OOD) performance gap observed in prior research on pre-trained Language Models (PLMs, such as BERT). We demonstrate this across two non-topical classification tasks: 1) genre classification and 2) generated text detection. Our results show that when demonstration examples for In-Context Learning (ICL) come from one domain (e.g., travel) and the system is tested on another domain (e.g., history), classification performance declines significantly. To address this, we introduce a method that controls which predictive indicators are used and which are excluded during classification. For the two tasks studied here, this ensures that topical features are omitted, while the model is guided to focus on stylistic rather than content-based attributes. This approach reduces the OOD gap by up to 20 percentage points in a few-shot setup. Straightforward Chain-of-Thought (CoT) methods, used as the baseline, prove insufficient, while our approach consistently enhances domain transfer performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。