arXiv:2508.04820cs.SEcs.AI2025-08被引 4

用大模型自动生成机器学习代码的文件级日志,效果不错但容易过度记录。

Automated File-Level Logging Generation for Machine Learning Applications using LLMs: A Case Study using GPT-4o Mini

  • 用GPT-4o mini在文件层面生成日志,模拟人类编写位置。
  • 63.91%位置准确,但82.66%出现过度日志,且内容质量参差。
  • 适合研究日志自动化与大模型在工程实践中的局限性。

日志在软件开发中至关重要,有助于监控系统行为并辅助调试。鉴于大语言模型(LLM)生成自然语言和代码的能力,研究者正探索其生成日志语句的潜力。然而,以往工作集中于函数级日志评估,对文件级日志生成,尤其是在机器学习(ML)应用中,仍缺乏研究——而全面的日志可提升系统可靠性。本研究以GPT-4o mini为案例,评估其在文件级别生成ML项目日志的能力。我们收集了171个包含4,073个Python文件的ML仓库,移除原有日志后,通过提示让LLM生成新日志,并对比生成日志的位置、日志等级、变量使用及文本质量。结果表明,LLM在63.91%的案例中将日志放置于与人类相似的位置,但存在82.66%的过量日志率。手动分析揭示常见问题:日志集中在函数首尾、难以在大代码块中精准插入,以及与项目特定日志规范不一致。尽管显示潜力,但实现落地仍需解决上述限制。

原文摘要 · Abstract (English)

Logging is essential in software development, helping developers monitor system behavior and aiding in debugging applications. Given the ability of large language models (LLMs) to generate natural language and code, researchers are exploring their potential to generate log statements. However, prior work focuses on evaluating logs introduced in code functions, leaving file-level log generation underexplored -- especially in machine learning (ML) applications, where comprehensive logging can enhance reliability. In this study, we evaluate the capacity of GPT-4o mini as a case study to generate log statements for ML projects at file level. We gathered a set of 171 ML repositories containing 4,073 Python files with at least one log statement. We identified and removed the original logs from the files, prompted the LLM to generate logs for them, and evaluated both the position of the logs and log level, variables, and text quality of the generated logs compared to human-written logs. In addition, we manually analyzed a representative sample of generated logs to identify common patterns and challenges. We find that the LLM introduces logs in the same place as humans in 63.91% of cases, but at the cost of a high overlogging rate of 82.66%. Furthermore, our manual analysis reveals challenges for file-level logging, which shows overlogging at the beginning or end of a function, difficulty logging within large code blocks, and misalignment with project-specific logging conventions. While the LLM shows promise for generating logs for complete files, these limitations remain to be addressed for practical implementation.

日志生成大模型机器学习自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。