识别并检测大模型代码中的五大低质量编码模式。
Specification and Detection of LLM Code Smells
- 定义了大模型推理中五类典型不良编码实践。
- 60.5%的开源系统存在这类问题，检测准确率达86.06%。
- 适合关注AI集成质量的开发者和代码审查人员。
近年来，大语言模型（LLMs）在软件系统中广泛应用，但其不当集成可能损害系统质量。然而，目前尚无针对大模型推理编码实践的正式代码异味（code smells）目录。本文首次提出LLM代码异味概念，并基于相关文献归纳出五类与大模型推理相关的常见不良编程实践。我们扩展了检测工具SpecDetect4AI以支持新定义的LLM代码异味，并在200个开源大模型系统数据集上验证其普遍性。结果显示，60.50%的系统存在此类问题，检测精度达86.06%。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have gained massive popularity in recent years and are increasingly integrated into software systems for diverse purposes. However, poorly integrating them in source code may undermine software system quality. Yet, to our knowledge, there is no formal catalog of code smells specific to coding practices for LLM inference. In this paper, we introduce the concept of LLM code smells and formalize five recurrent problematic coding practices related to LLM inference in software systems, based on relevant literature. We extend the detection tool SpecDetect4AI to cover the newly defined LLM code smells and use it to validate their prevalence in a dataset of 200 open-source LLM systems. Our results show that LLM code smells affect 60.50% of the analyzed systems, with a detection precision of 86.06%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。