用词频特征和梯度提升树检测AI生成文本，速度快且可解释。
StylOch at PAN: Gradient-Boosted Trees with Frequency-Based Stylometric Features
- 基于spaCy提取语言标注频次，生成数千个风格特征
- 在50万+机器生成文本上训练，准确率显著提升
- 非神经网络方法，适合需要透明决策的场景
本研究针对二分类的AI文本检测任务，构建了一个模块化的风格特征分析流程：使用公开的spaCy模型进行文本预处理（包括分词、命名实体识别、依存句法分析、词性标注和形态学标注），并提取上述语言标注的n-gram频次，生成数千个特征；采用轻量级梯度提升机作为分类器。为训练分类器，收集了超过50万条机器生成文本组成的大型语料库。通过探索多种参数配置，充分发挥该数据集的潜力。该方法延续了此前被证明有效的非神经网络、计算开销低但可解释性强的思路。
原文摘要 · Abstract (English)
This submission to the binary AI detection task is based on a modular stylometric pipeline, where: public spaCy models are used for text preprocessing (including tokenisation, named entity recognition, dependency parsing, part-of-speech tagging, and morphology annotation) and extracting several thousand features (frequencies of n-grams of the above linguistic annotations); light-gradient boosting machines are used as the classifier. We collect a large corpus of more than 500 000 machine-generated texts for the classifier's training. We explore several parameter options to increase the classifier's capacity and take advantage of that training set. Our approach follows the non-neural, computationally inexpensive but explainable approach found effective previously.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。