用自动生成的密码学竞赛题训练大模型，提升其解题能力。
Improving LLM Agents with Reinforcement Learning on Cryptographic CTF Challenges
- 基于程序生成的密码学题目，用强化学习微调大模型。
- 在未见过的题目上，通过率提升显著，且跨任务泛化有效。
- 适合想提升安全领域智能体能力的研究者和开发者。
我们提出'Random-Crypto'，一个程序生成的密码学夺旗（CTF）数据集，旨在激发强化学习在安全敏感领域的大型语言模型（LLM）代理中的潜力。密码推理提供了理想的强化学习测试环境：具备精确验证、结构化的多步推理过程，以及对可靠计算工具使用的依赖。利用这些特性，我们在安全执行环境中，通过组相对策略优化（GRPO）对增强Python工具的Llama-3.1-8B进行微调。结果表明，该代理在先前未见挑战上的Pass@8指标显著提升。此外，性能改进可泛化至两个外部基准：涵盖密码与非密码任务的'picoCTF'，以及包含135道密码学选择题的'AICrypto MCQ'。消融实验表明，性能提升主要源于工具使用能力和程序化推理的增强。这些发现使'Random-Crypto'成为构建能够应对复杂网络安全任务的智能、适应性强的LLM代理的重要训练平台。
原文摘要 · Abstract (English)
We present 'Random-Crypto', a procedurally generated cryptographic Capture The Flag (CTF) dataset designed to unlock the potential of Reinforcement Learning (RL) for LLM-based agents in security-sensitive domains. Cryptographic reasoning offers an ideal RL testbed: it combines precise validation, structured multi-step inference, and reliance on reliable computational tool use. Leveraging these properties, we fine-tune a Python tool-augmented Llama-3.1-8B via Group Relative Policy Optimization (GRPO) in a secure execution environment. The resulting agent achieves a significant improvement in Pass@8 on previously unseen challenges. Moreover, the improvements generalize to two external benchmarks: 'picoCTF', spanning both crypto and non-crypto tasks, and 'AICrypto MCQ', a multiple-choice benchmark of 135 cryptography questions. Ablation studies attribute the gains to enhanced tool usage and procedural reasoning. These findings position 'Random-Crypto' as a rich training ground for building intelligent, adaptable LLM agents capable of handling complex cybersecurity tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。