arXiv:2504.15254cs.SEcs.CL2025-04被引 31

构建首个完整C转安全Rust的评测基准,评估代码迁移效果。

CRUST-Bench: A Comprehensive Benchmark for C-to-safe-Rust Transpilation

  • 用100个真实C项目+手动编写Rust接口与测试用例构建评测集
  • 顶尖模型单次尝试仅能正确转换15个任务,仍存严重缺陷
  • 适合研究代码翻译、LLM编程能力及系统化迁移工具的开发者

C转Rust的代码移植对现代化遗留C代码至关重要,可提升安全性并增强与现代Rust生态的互操作性。然而,目前尚无数据集用于评估系统是否能将C代码成功转换为通过测试用例验证的安全Rust代码。我们提出CRUST-Bench,包含100个C代码仓库,每个仓库均配有手工编写的安全Rust接口和可用于验证转换正确性的测试用例。通过考虑整个仓库而非孤立函数,该数据集捕捉了跨文件依赖等复杂项目的转换挑战。提供的Rust接口明确了符合习惯且内存安全的模式,测试用例则确保功能正确性。我们在该任务上评估了当前最先进的大语言模型(LLMs),发现生成安全且符合习惯的Rust代码仍是重大挑战。表现最佳的模型OpenAI o1在单次设置下仅能解决15个任务。对CRUST-Bench的改进将推动更强大代码迁移系统的发展,助力从C向保证内存安全的语言如Rust的代码库迁移。数据集与代码可在https://github.com/anirudhkhatry/CRUST-bench获取。

原文摘要 · Abstract (English)

C-to-Rust transpilation is essential for modernizing legacy C code while enhancing safety and interoperability with modern Rust ecosystems. However, no dataset currently exists for evaluating whether a system can transpile C into safe Rust that passes a set of test cases. We introduce CRUST-Bench, a dataset of 100 C repositories, each paired with manually-written interfaces in safe Rust as well as test cases that can be used to validate correctness of the transpilation. By considering entire repositories rather than isolated functions, CRUST-Bench captures the challenges of translating complex projects with dependencies across multiple files. The provided Rust interfaces provide explicit specifications that ensure adherence to idiomatic, memory-safe Rust patterns, while the accompanying test cases enforce functional correctness. We evaluate state-of-the-art large language models (LLMs) on this task and find that safe and idiomatic Rust generation is still a challenging problem for various state-of-the-art methods and techniques. We also provide insights into the errors LLMs usually make in transpiling code from C to safe Rust. The best performing model, OpenAI o1, is able to solve only 15 tasks in a single-shot setting. Improvements on CRUST-Bench would lead to improved transpilation systems that can reason about complex scenarios and help in migrating legacy codebases from C into languages like Rust that ensure memory safety. You can find the dataset and code at https://github.com/anirudhkhatry/CRUST-bench.

代码迁移RustLLM编程安全编程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。