提出方法验证代码库中AI模式的实际使用情况
A Methodology for Investigating AI Patterns Prevalence in Software Repositories

- 从44篇文献中提取14类AI模式,用主动学习检测真实代码
- 在100个GitHub项目中发现模式普遍存在,8分类准确率达56%
- 为理解AI模式实践应用提供可量化的分析框架
随着基于人工智能的应用迅速发展,理解AI模式对提升应用质量至关重要。尽管已有大量文献提出各种AI模式,但其在实际代码中的普遍性尚未得到验证。本文提出一种方法论:首先通过文献挖掘识别14类AI模式(基于44篇相关文献),再利用主动学习在100个GitHub开源AI项目中验证这些模式的出现频率。通过流行度估计,我们给出了出现次数的置信区间。模型在8分类任务中达到56%准确率和55%召回率,显著优于11%的随机基线。该方法为实证研究AI模式在实际中的应用提供了可靠基础,填补了该领域缺乏数据支持的空白。
原文摘要 · Abstract (English)
As Artificial Intelligence(AI)-based applications take off, a clear understanding of AI patterns can uplift the quality of AI applications. Many AI patterns have been proposed in the literature; however, their prevalence in real-life code has not yet been validated. Understanding the actual use of those patterns in practice can clarify our understanding both of the significance of these patterns and their utility. In this paper, we present a methodology to a) identify relevant patterns by mining the literature and then to b) validate their presence and prevalence in actual code repositories using active learning. To that end, we identify 14 AI pattern classes by mining 44 published AI pattern-related sources. Then we use an active learning approach to determine the prevalence of the most common pattern class across 100 GitHub open AI repositories. Using prevalence estimation, we propose bounds on the accuracy of the occurrences. The model achieves 56\% accuracy and 55\% recall in an 8-way classification task, significantly outperforming the 11\% random-chance baseline. Furthermore, the prevalence estimation offers usable bounds for analyzing pattern applications. This methodology provides a robust foundation to start understanding how AI patterns are used in practice, a field that currently lacks empirical data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。