多任务学习提升生成文本检测，首获竞赛冠军
Advacheck at GenAI Detection Task 1: AI Detection Powered by Domain-Aware Multi-Tasking
- 共享编码器的多任务架构，同时完成真假文本与领域分类
- 测试集达83.07%宏F1，超越基线10个百分点
- 领域分类帮助理解样本，嵌入空间呈现聚类结构
本文介绍Advacheck团队为GenAI Detection Task 1单语子任务设计的文本生成检测系统。该系统采用多任务架构，共享Transformer编码器，配备多个分类头：一个负责人类写作与机器生成文本的二分类，其余为辅助多分类头，用于识别不同数据集中的文本领域。由于多分类头在训练中区分了数据中的具体领域，从而增强了对样本的理解。该方法在官方排名中取得第一名，测试集上宏F1得分为83.07%，较基线提升10%。进一步通过消融实验、错误分析和表示分析发现，多任务学习优于单任务模式，且各任务在嵌入空间中形成聚类结构。
原文摘要 · Abstract (English)
The paper describes a system designed by Advacheck team to recognise machine-generated and human-written texts in the monolingual subtask of GenAI Detection Task 1 competition. Our developed system is a multi-task architecture with shared Transformer Encoder between several classification heads. One head is responsible for binary classification between human-written and machine-generated texts, while the other heads are auxiliary multiclass classifiers for texts of different domains from particular datasets. As multiclass heads were trained to distinguish the domains presented in the data, they provide a better understanding of the samples. This approach led us to achieve the first place in the official ranking with 83.07% macro F1-score on the test set and bypass the baseline by 10%. We further study obtained system through ablation, error and representation analyses, finding that multi-task learning outperforms single-task mode and simultaneous tasks form a cluster structure in embeddings space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。