研究学生如何用特殊指令骗过AI编程评分系统,揭示其严重漏洞。
How to Trick Your AI TA: A Systematic Study of Academic Jailbreaking in LLM Code Evaluation
- 设计20多种学术场景下的对抗性提示策略,形成新攻击类型。
- 在25,000条伪造作业数据上测试,最高欺骗成功率达97%。
- 提出三类评估指标,适合安全与教育领域研究人员参考。
大型语言模型(LLMs)作为代码自动评分工具在学术环境中日益普及,但其可靠性可能被学生通过对抗性提示策略破坏,从而获得不当分数优势。本文首次对学术场景下基于LLM的代码评价器进行大规模研究。贡献包括:(i) 系统性地适配20余种越狱策略,定义了一类新的攻击形式——学术越狱;(ii) 发布一个包含2.5万条对抗性学生提交的污染数据集,数据源自多样化的实际课程作业,并配有评分标准和人工评分参考;(iii) 为捕捉学术越狱的多维影响,系统性地定义了三项评估指标:越狱成功率(JSR)、分数膨胀率与危害性;(iv) 使用六种LLM全面评估这些攻击。结果表明,这些模型表现出显著脆弱性,尤其在说服型和角色扮演型攻击下(最高JSR达97%)。该对抗数据集与基准套件为下一代鲁棒的学术代码评估系统奠定了基础。
原文摘要 · Abstract (English)
The use of Large Language Models (LLMs) as automatic judges for code evaluation is becoming increasingly prevalent in academic environments. But their reliability can be compromised by students who may employ adversarial prompting strategies in order to induce misgrading and secure undeserved academic advantages. In this paper, we present the first large-scale study of jailbreaking LLM-based automated code evaluators in academic context. Our contributions are: (i) We systematically adapt 20+ jailbreaking strategies for jailbreaking AI code evaluators in the academic context, defining a new class of attacks termed academic jailbreaking. (ii) We release a poisoned dataset of 25K adversarial student submissions, specifically designed for the academic code-evaluation setting, sourced from diverse real-world coursework and paired with rubrics and human-graded references, and (iii) In order to capture the multidimensional impact of academic jailbreaking, we systematically adapt and define three jailbreaking metrics (Jailbreak Success Rate, Score Inflation, and Harmfulness). (iv) We comprehensively evalulate the academic jailbreaking attacks using six LLMs. We find that these models exhibit significant vulnerability, particularly to persuasive and role-play-based attacks (up to 97% JSR). Our adversarial dataset and benchmark suite lay the groundwork for next-generation robust LLM-based evaluators in academic code assessment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。