针对印地语等三大印度语言,构建了真实场景下的文本生成检测基准。
IndicDetect: Evaluating Cross-Lingual LLM-Generated Text Detection for Hindi, Telugu, and Tamil

- 构建涵盖多领域、多生成器的跨语言检测数据集
- 发现现有检测器在对抗攻击下性能大幅下降,尤其印地语最严重
- 为印度文字的生成文本检测提供标准化评估框架和基线
大型语言模型的快速普及进一步加剧了对可靠人工智能生成文本检测的需求,尤其是在英语以外的语言中。然而,当前基准对印地语系语言关注极少,且测试环境过于理想化,无法反映真实情况。我们提出了一个面向印地语、泰卢固语和泰米尔语的通用检测基准——IndicDetect,旨在评估检测器在真实分布偏移下的鲁棒性。IndicDetect包含经过精心筛选的人类撰写文本及其对应的LLM生成版本,覆盖多个领域与生成器,并系统评估检测器在领域偏移、生成器偏移及对抗扰动下的表现。采用统一可复现的评估方案,我们测试了多种统计与神经检测方法。结果显示存在显著鲁棒性失效:监督型神经检测器在分布内表现良好,而无训练方法在未见生成器和对抗攻击下性能急剧下降。各语言间差异明显,印地语在对抗扰动下退化最严重。这些结果表明,现有检测器在印地语场景中的主要弱点在于鲁棒性,而非峰值准确率。IndicDetect提供了标准数据划分、评估协议与基线,为印地语书写系统的生成文本检测奠定坚实、语言敏感的基础。
原文摘要 · Abstract (English)
The rapid proliferation of LLMs has further heightened the need to develop dependable AI-generated text detection, especially beyond English. Nevertheless, current benchmarks pay little attention to Indic languages and test detectors in idealized settings that do not represent the real world. We present a generalized benchmark for AI-generated text detection in Hindi, Telugu, and Tamil, which we call IndicDetect, designed to assess the robustness of detectors under realistic distribution shifts. IndicDetect comprises highly curated human-written texts matched with LLM-generated counterparts across various domains and generators, and systematically evaluates detectors in the presence of domain shift, generator shift, and adversarial perturbation. Using a single and repeatable evaluation scheme, we evaluate a wide range of statistical and neural detectors. We find substantial robustness failures: supervised neural detectors perform well in-distribution, while training-free methods degrade considerably under unseen generators and adversarial attacks. The severity of these failures varies across languages, with Hindi exhibiting the largest overall degradation under adversarial perturbations. These results highlight that the primary weakness of existing detectors in Indic settings lies in their robustness, not in their peak accuracy. IndicDetect provides standard data splits, an evaluation protocol, and baselines to establish a robust, language-aware foundation for AI-generated text detection in Indic scripts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。