通过贝叶斯混合数据与经验风险最小化，提升AI文本检测在分布外数据上的性能。
Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection
- 用贝叶斯分类头筛选三组数据，构建融合训练集以减少过拟合。
- ModernBERT-large模型在五个指标上平均得分0.96，MCGrad达0.974，表现最优。
- 适合关注AI文本检测泛化能力的研究者与开发者参考。
现有研究显示，AI生成文本检测分类器在分布内（ID）数据上表现良好，但在分布外（OOD）数据上性能下降，表明对数据集特定特征存在过拟合。然而，简单合并不同训练数据集并不总能提升性能，甚至可能引发捷径学习。为此，我们采用贝叶斯分类头微调BERT-tiny模型，从三个不同数据集中选择文本构建整合训练集。训练了三种分类器：通过经验X风险最小化微调的DeBERTa-V3-large和ModernBERT-large，以及基于ModernBERT-large预测进行校准的MCGrad模型。DeBERTa-V3-large在PAN 2026测试集的五个指标（AUROC、$F_1$、C@1、Brier score、$F_{0.5u}$）上平均得分为0.882；ModernBERT-large得分为0.96；MCGrad达到三者最高分0.974，在排行榜中位列第二。结果表明，精心的数据集筛选可显著提升模型在分布外场景下的性能。相关模型已发布于https://huggingface.co/collections/ShantanuT01/panclef-2026。
原文摘要 · Abstract (English)
Existing research shows that AI-generated text detection classifiers achieve strong in-distribution (ID) performance but do not maintain the same performance on out-of-distribution (OOD) texts, suggesting overfitting to dataset-specific features. However, combining different training datasets doesn't always improve performance and, in some cases, can even encourage shortcut learning. To address this issue, we fine-tune BERT-tiny models with Bayesian classification heads to select texts across three different datasets to use as a consolidated training set. We trained three different classifiers: fine-tuned DeBERTa-V3-large and ModernBERT-large classifiers via empirical X-risk minimization, and an MCGrad model that calibrates the predictions from the ModernBERT-large classifier. The DeBERTa-V3-large-large classifier achieves a mean score of 0.882 on the PAN 2026 test set across five metrics: AUROC, $F_1$, C@1, Brier score, and $F_{0.5u}$. ModernBERT-large achieves a score of 0.96 while MCGrad achieves the best score of the three with a mean score of 0.974, ranking second on the leaderboard. Our results highlight that careful dataset curation can lead to strong OOD performance. We release our ModernBERT-large and DeBERTa-V3-large models at https://huggingface.co/collections/ShantanuT01/panclef-2026 .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。