用可解释方法复现并改进了机器文本作者识别系统。
Interpretable Predictability-Based AI Text Detection: A Replication Study
- 采用多语言模型与26个文档级风格特征，结合可解释性分析。
- 在英西双语任务中表现优于或相当语言专用基线，尤其在模型归属上提升明显。
- 强调清晰文档对复现性和公平比较的重要性，适合关注可解释AI的读者。
本文复现并扩展了用于机器生成文本作者归属的AuTexTification共享任务系统。由于数据划分、模型可用性和实现细节差异，完全复现不可行，我们将其作为可复现性案例进行记录。测试了mDeBERTa-v3-base、Qwen、mGPT等新多语言模型，并新增26个文档级风格特征，通过消融实验、置换重要性及SHAP分析评估特征影响。单一共享配置在英语和西班牙语的子任务1和子任务2上均适用。三次随机种子平均下，该配置表现相当于或优于语言特定基线，尤其在模型归属（子任务2）中提升显著。新增风格特征带来小幅改进，以词汇多样性为先，但一旦引入基于可预测性的概率，其贡献即被覆盖于种子方差内，该信号仍为主导。研究还表明，详尽文档对可靠复现和系统公平比较至关重要。
原文摘要 · Abstract (English)
This paper replicates and extends the system used in the AuTexTification shared task for authorship attribution of machine-generated texts. Exact replication was not possible because of differences in data splits, model availability, and implementation details, which we document as a case study in reproducibility. We tested newer multilingual language models (mDeBERTa-v3-base, Qwen, mGPT) and added 26 document-level stylometric features, using ablation, permutation importance, and SHAP analysis to assess feature influence. A single shared configuration was applied to both English and Spanish across Subtask 1 and Subtask 2. Averaged over three random seeds, the shared multilingual configuration performs comparably to or better than the language-specific baseline, with the clearest gains on model attribution (Subtask 2). The additional stylometric features yield small improvements, led by lexical diversity, but their contribution falls within seed variance once predictability-based probabilities are included, which remain the dominant signal. The study also shows that clear documentation is important for reliable replication and fair comparison of systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。