arXiv:2506.02046cs.CYcs.AI2025-06被引 2

用AI对抗AI生成内容,构建评估防伪新框架

Machine vs Machine: Using AI to Tackle Generative AI Threats in Assessment

  • 提出机器对机器的双策略框架,结合静态分析与动态测试
  • 8项人类特有能力要素可有效识别AI生成内容,避免误判
  • 适合教育机构和考试设计者应对生成式AI威胁

本文提出一种理论框架,以应对生成式人工智能(如GPT-4、Claude、Llama)在高等教育评估中带来的挑战。当前74%-92%的学生已尝试使用这些工具完成学术任务,传统评估方式面临生存危机。现有应对措施如检测软件存在对非母语者偏见,且易被规避;人工重设计依赖主观判断,且假设AI能力不变。本文提出双策略范式:静态分析包含八项理论合理要素——具体性与情境化、时间相关性、过程可见性要求、个性化元素、资源可及性、多模态融合、伦理推理要求、协作要素,每项均针对生成式AI的能力短板,形成区分真实学习与模拟输出的屏障;动态测试则通过模拟攻击进行漏洞评估,弥补基于模式分析的局限。论文还提出漏洞评分的理论基础,包括量化评估方法、权重设定及阈值确定理论。

原文摘要 · Abstract (English)

This paper presents a theoretical framework for addressing the challenges posed by generative artificial intelligence (AI) in higher education assessment through a machine-versus-machine approach. Large language models like GPT-4, Claude, and Llama increasingly demonstrate the ability to produce sophisticated academic content, traditional assessment methods face an existential threat, with surveys indicating 74-92% of students experimenting with these tools for academic purposes. Current responses, ranging from detection software to manual assessment redesign, show significant limitations: detection tools demonstrate bias against non-native English writers and can be easily circumvented, while manual frameworks rely heavily on subjective judgment and assume static AI capabilities. This paper introduces a dual strategy paradigm combining static analysis and dynamic testing to create a comprehensive theoretical framework for assessment vulnerability evaluation. The static analysis component comprises eight theoretically justified elements: specificity and contextualization, temporal relevance, process visibility requirements, personalization elements, resource accessibility, multimodal integration, ethical reasoning requirements, and collaborative elements. Each element addresses specific limitations in generative AI capabilities, creating barriers that distinguish authentic human learning from AI-generated simulation. The dynamic testing component provides a complementary approach through simulation-based vulnerability assessment, addressing limitations in pattern-based analysis. The paper presents a theoretical framework for vulnerability scoring, including the conceptual basis for quantitative assessment, weighting frameworks, and threshold determination theory.

AI评估生成式AI教育安全检测框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。