用特征增强提升AI文本检测模型跨领域鲁棒性
Feature-Augmented Transformers for Robust AI-Text Detection Across Domains and Generators
- 通过注意力融合语言特征增强Transformer模型
- 在跨数据集测试中达到85.9%的平衡准确率
- 适合关注实际部署中检测器泛化能力的研究者
当前大规模生成的AI文本分布于不同领域和生成管道,使检测模型对分布偏移的鲁棒性成为关键。研究在HC3 PLUS上训练基于Transformer的检测器,并通过验证集最大化平衡准确率确定单一固定决策阈值,该阈值在所有下游测试分布中保持不变,揭示了分布偏移下的领域与生成器依赖误差不对称性。在HC3 PLUS进行域内评估,在多领域多生成器的M4基准上进行跨数据集迁移,以及在外部的AI-Text-Detection-Pile上测试。尽管基础模型在域内表现接近天花板(最高99.5%平衡准确率),但在分布偏移下性能脆弱且高度依赖模型。通过基于注意力的语言特征融合进行特征增强,显著提升迁移性能,最优模型DeBERTa-v3-base+FeatAttn在M4上达到85.9%平衡准确率。多种子实验验证了高稳定性。在相同固定阈值协议下,该模型相比强零样本基线最高提升7.22个百分点。类别级消融显示,可读性和词汇特征对偏移下的鲁棒性贡献最大。结果表明,特征增强与现代DeBERTa骨干网络显著优于早期BERT/RoBERTa模型,而固定阈值协议为实际检测器鲁棒性提供了更真实、更有信息量的评估。
原文摘要 · Abstract (English)
AI-generated text is nowadays produced at scale across domains and heterogeneous generation pipelines, making robustness to distribution shift a central requirement for supervised binary detectors. We train transformer-based detectors on HC3 PLUS and calibrate a single decision threshold by maximising balanced accuracy on held-out validation; this threshold is then kept fixed for all downstream test distributions, revealing domain- and generator-dependent error asymmetries under shift. We evaluate in-domain on HC3 PLUS, under cross-dataset transfer to the multi-domain, multi-generator M4 benchmark, and on the external AI-Text-Detection-Pile. Although base models achieve near-ceiling in-domain performance (up to 99.5% balanced accuracy), performance under shift is brittle and strongly model-dependent. Feature augmentation via attention-based linguistic feature fusion improves transfer, with our best model (DeBERTa-v3-base+FeatAttn) achieving 85.9% balanced accuracy on M4. Multi-seed experiments confirm high stability. Under the same fixed-threshold protocol, our model outperforms strong zero-shot baselines by up to +7.22 points. Category-level ablations further show that readability and vocabulary features contribute most to robustness under shift. Overall, these results demonstrate that feature augmentation and a modern DeBERTa backbone significantly outperform earlier BERT/RoBERTa models, while the fixed-threshold protocol provides a more realistic and informative assessment of practical detector robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。