轻量级模型可精准区分假新闻与讽刺内容，适合实际部署。
WISE: Web Information Satire and Fakeness Evaluation
- 构建 WISE 框架，对比 8 个轻量 Transformer 与 2 个基线模型
- MiniLM 准确率达 87.58%，RoBERTa-base ROC-AUC 达 95.42%
- 轻量模型在效率与性能间平衡，适合资源受限场景
区分假新闻与讽刺内容因语言特征重叠、意图差异而极具挑战。本研究提出 WISE（Web Information Satire and Fakeness Evaluation）框架，在包含 20,000 条样本的 Fakeddit 数据集上，对 8 个轻量级 Transformer 模型与 2 个基线模型进行评估，样本标注为假新闻或讽刺内容。采用分层 5 折交叉验证，全面评估准确率、精确率、召回率、F1 值、ROC-AUC、PR-AUC、MCC、Brier 分数与期望校准误差。结果显示，MiniLM 在所有模型中达到最高准确率（87.58%），RoBERTa-base 在 ROC-AUC 上表现最优（95.42%），同时保持 87.36% 准确率；DistilBERT 在准确率（86.28%）与 ROC-AUC（93.90%）间实现良好平衡。统计检验（配对 t 检验与 McNemar 检验）证实模型间性能差异显著。研究表明，轻量级模型可媲美甚至超越基线，为资源受限环境下的虚假信息检测系统提供可行方案。
原文摘要 · Abstract (English)
Distinguishing fake or untrue news from satire or humor poses a unique challenge due to their overlapping linguistic features and divergent intent. This study develops WISE (Web Information Satire and Fakeness Evaluation) framework which benchmarks eight lightweight transformer models alongside two baseline models on a balanced dataset of 20,000 samples from Fakeddit, annotated as either fake news or satire. Using stratified 5-fold cross-validation, we evaluate models across comprehensive metrics including accuracy, precision, recall, F1-score, ROC-AUC, PR-AUC, MCC, Brier score, and Expected Calibration Error. Our evaluation reveals that MiniLM, a lightweight model, achieves the highest accuracy (87.58%) among all models, while RoBERTa-base achieves the highest ROC-AUC (95.42%) and strong accuracy (87.36%). DistilBERT offers an excellent efficiency-accuracy trade-off with 86.28\% accuracy and 93.90\% ROC-AUC. Statistical tests confirm significant performance differences between models, with paired t-tests and McNemar tests providing rigorous comparisons. Our findings highlight that lightweight models can match or exceed baseline performance, offering actionable insights for deploying misinformation detection systems in real-world, resource-constrained settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。