比较人与机器识别钓鱼邮件的能力和信心，发现人类更稳定、用更多语言线索。
Evaluating Human and Machine Confidence in Phishing Email Detection: A Comparative Study
- 用逻辑回归、决策树、随机森林结合文本特征与语义嵌入
- 模型准确率高但信心波动大，人类更一致且使用更多语言线索
- 年龄影响判断力，语言水平影响小，适合人机协作场景
识别钓鱼邮件等欺骗性内容需要复杂的认知过程，包括模式识别、置信度评估和上下文分析。本研究考察了人类认知与机器学习模型在区分钓鱼邮件与正常邮件中的协同作用。采用三种可解释算法——逻辑回归、决策树和随机森林，基于TF-IDF特征与语义嵌入进行训练，并将其预测结果与人类评估（包含置信度评分和语言观察）进行对比。结果显示，机器学习模型具有较高的准确率，但其置信度波动显著；而人类评估者使用更丰富的语言标志，且置信度更为稳定。此外，语言能力对检测表现影响较小，但年龄有显著影响。这些发现为构建透明的AI系统以补充人类认知功能提供了有益方向，有助于提升复杂内容分析任务中的人机协作效果。
原文摘要 · Abstract (English)
Identifying deceptive content like phishing emails demands sophisticated cognitive processes that combine pattern recognition, confidence assessment, and contextual analysis. This research examines how human cognition and machine learning models work together to distinguish phishing emails from legitimate ones. We employed three interpretable algorithms Logistic Regression, Decision Trees, and Random Forests training them on both TF-IDF features and semantic embeddings, then compared their predictions against human evaluations that captured confidence ratings and linguistic observations. Our results show that machine learning models provide good accuracy rates, but their confidence levels vary significantly. Human evaluators, on the other hand, use a greater variety of language signs and retain more consistent confidence. We also found that while language proficiency has minimal effect on detection performance, aging does. These findings offer helpful direction for creating transparent AI systems that complement human cognitive functions, ultimately improving human-AI cooperation in challenging content analysis tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。