针对大模型生成文本检测稳定性差的问题,提出混合集成方法提升泛化能力。
Simplex-Optimized Hybrid Ensemble for Large Language Model Text Detection Under Generative Distribution Drif
- 融合三种互补模型:微调的RoBERTa分类器、基于输入扰动的曲率得分、手工特征的风格模型。
- 在3万篇文档上达到94.2%准确率和0.978 AUC,对未见模型和改写攻击均有效。
- 通过概率单纯形加权融合,降低误报率,适合教育科研场景使用。
大语言模型的广泛应用使得区分人类写作与机器生成文本变得困难。针对旧检测器在新模型或解码策略下性能下降的问题,本文研究了生成分布变化带来的不稳定性,并提出一种显式应对分布漂移的混合集成方法。该集成结合三个互补组件:基于RoBERTa的微调分类器、基于输入扰动与模型似然变化的曲率得分、基于手工语言特征的紧凑风格模型。各组件输出在概率单纯形上融合,权重通过验证集搜索确定。方法从方差缩减与混合生成器风险角度建模,证明单纯形约束可灵活平衡各分支优劣。在包含多个大模型家族的3万篇文档数据集上测试,涵盖训练中未见模型和改写攻击变体,本方法实现94.2%准确率和0.978 AUC。相比强基线,该方法显著降低科学论文中的误报率,对教育与科研场景中避免误标人类作品具有重要意义。
原文摘要 · Abstract (English)
The widespread adoption of large language models (LLMs) has made it difficult to distinguish human writing from machine-produced text in many real applications. Detectors that were effective for one generation of models tend to degrade when newer models or modified decoding strategies are introduced. In this work, we study this lack of stability and propose a hybrid ensemble that is explicitly designed to cope with changing generator distributions. The ensemble combines three complementary components: a RoBERTa-based classifier fine-tuned for supervised detection, a curvature-inspired score based on perturbing the input and measuring changes in model likelihood, and a compact stylometric model built on hand-crafted linguistic features. The outputs of these components are fused on the probability simplex, and the weights are chosen via validation-based search. We frame this approach in terms of variance reduction and risk under mixtures of generators, and show that the simplex constraint provides a simple way to trade off the strengths and weaknesses of each branch. Experiments on a 30000 document corpus drawn from several LLM families including models unseen during training and paraphrased attack variants show that the proposed method achieves 94.2% accuracy and an AUC of 0.978. The ensemble also lowers false positives on scientific articles compared to strong baselines, which is critical in educational and research settings where wrongly flagging human work is costly
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。