arXiv:2412.08061cs.SEcs.AI2024-12

用AI自动判断Go并发程序测试通过与否,解决人工判别难问题。

Go-Oracle: Automated Test Oracle for Go Concurrency Bugs

  • 收集并编码Go程序的执行轨迹,用Transformer模型分类成败
  • 在8个真实项目上测试,小数据集下准确率高
  • 适合需要自动化测试验证的Go语言开发者

Go语言在基础设施系统中广泛应用,但其双并发机制(通信顺序进程与共享内存)导致并发错误频发,测试结果的通过/失败判定极为困难,即使对领域专家也构成挑战。我们对字节跳动的开发人员调研确认了这一难题。本文提出Go-Oracle,通过采集多个目标Go程序的通过与失败执行轨迹,利用原生Go执行追踪器捕获全面事件,经预处理与编码后,训练基于Transformer的神经网络以自动分类测试执行结果。评估覆盖来自GoBench仓库的8个真实项目,这些项目在工业界常作为基准。实验表明,即使在有限数据下,Go-Oracle仍展现出高准确率,字节跳动开发者普遍表示愿用该工具替代当前的人工检查流程。

原文摘要 · Abstract (English)

The Go programming language has gained significant traction for developing software, especially in various infrastructure systems. Nonetheless, concurrency bugs have become a prevalent issue within Go, presenting a unique challenge due to the language's dual concurrency mechanisms-communicating sequential processes and shared memory. Detecting concurrency bugs and accurately classifying program executions as pass or fail presents an immense challenge, even for domain experts. We conducted a survey with expert developers at Bytedance that confirmed this challenge. Our work seeks to address the test oracle problem for Go programs, to automatically classify test executions as pass or fail. This problem has not been investigated in the literature for Go programs owing to its distinctive programming model. Our approach involves collecting both passing and failing execution traces from various subject Go programs. We capture a comprehensive array of execution events using the native Go execution tracer. Subsequently, we preprocess and encode these traces before training a transformer-based neural network to effectively classify the traces as either passing or failing. The evaluation of our approach encompasses 8 subject programs sourced from the GoBench repository. These subject programs are routinely used as benchmarks in an industry setting. Encouragingly, our test oracle, Go-Oracle, demonstrates high accuracies even when operating with a limited dataset, showcasing the efficacy and potential of our methodology. Developers at Bytedance strongly agreed that they would use the Go-Oracle tool over the current practice of manual inspections to classify tests for Go programs as pass or fail.

并发测试AI判别Go语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。