用马尔可夫模型校准文本生成检测分数,提升准确性
Beyond Raw Detection Scores: Markov-Informed Calibration for Boosting Machine-Generated Text Detection
- 基于马尔可夫随机场建模上下文得分的两个规律,实现轻量校准
- 在跨大模型和改写攻击场景下,检测准确率显著优于基线
- 可无缝嵌入现有检测器,计算开销极低,适合实际部署
机器生成文本虽便捷,却带来虚假信息与钓鱼风险,亟需可靠检测。基于度量的方法因无需复杂模型而更实用,但存在生成过程随机性导致的标记级分数偏差问题。本文首次将代表性度量方法统一于框架中,揭示上下文得分的两个关键规律:邻近相似性与初始不稳定性。据此提出马尔可夫信息校准策略,利用马尔可夫随机场建模并以平均场近似实现轻量集成。在跨大模型、改写攻击等真实场景的实验中,该方法显著优于基线,且计算开销可忽略。代码已开源:https://github.com/tmlr-group/MRF_Calibration。
原文摘要 · Abstract (English)
While machine-generated texts (MGTs) offer great convenience, they also pose risks such as disinformation and phishing, highlighting the need for reliable detection. Metric-based methods, which extract statistically distinguishable features of MGTs, are often more practical than complex model-based methods that are prone to overfitting. Given their diverse designs, we first place representative metric-based methods within a unified framework, enabling a clear assessment of their advantages and limitations. Our analysis identifies a core challenge across these methods: the token-level detection score is easily biased by the inherent randomness of the MGTs generation process. To address this, we theoretically and empirically reveal two relationships of context detection scores that may aid calibration: Neighbor Similarity and Initial Instability. We then propose a Markov-informed score calibration strategy that models these relationships using Markov random fields, and implements it as a lightweight component via a mean-field approximation, allowing our method to be seamlessly integrated into existing detectors. Extensive experiments in various real-world scenarios, such as cross-LLM and paraphrasing attacks, demonstrate significant gains over baselines with negligible computational overhead. The code is available at https://github.com/tmlr-group/MRF_Calibration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。