研究大模型推荐的Python库是否可用，发现4.6%无法自动安装。
How Robust are LLM-Generated Library Imports? An Empirical Study using Stack Overflow
- 用Stack Overflow真实问题测试6个大模型的库推荐能力。
- 4.6%推荐库因名称不匹配导致无法自动安装，仅2个模型提供安装指引。
- 适合关注代码生成可靠性的开发者和研究者参考。
软件库是现代代码功能、安全性和可维护性的核心。随着开发者越来越多地使用大型语言模型（LLMs）辅助编程，理解这些模型如何推荐库至关重要。本文通过提示六种先进的大模型（包括专有和开源模型）解决来自Stack Overflow的真实Python问题，分析其推荐的库类型、特征以及推荐结果的可用性。结果表明，大模型倾向于推荐第三方库，且多为成熟、流行、许可宽松的依赖。然而，仍存在可用性缺口：4.6%的库因导入名与可安装包不匹配而无法自动解析，且仅有两个模型提供了安装指导。尽管生成代码技术上正确，但缺乏上下文支持，使用户需手动解决依赖问题。研究为开发者和研究人员提供了可操作的洞察，并指出了提升大模型生成代码在依赖管理方面可靠性的机会。
原文摘要 · Abstract (English)
Software libraries are central to the functionality, security, and maintainability of modern code. As developers increasingly turn to Large Language Models (LLMs) to assist with programming tasks, understanding how these models recommend libraries is essential. In this paper, we conduct an empirical study of six state-of-the-art LLMs, both proprietary and open-source, by prompting them to solve real-world Python problems sourced from Stack Overflow. We analyze the types of libraries they import, the characteristics of those libraries, and the extent to which the recommendations are usable out of the box. Our results show that LLMs predominantly favour third-party libraries over standard ones, and often recommend mature, popular, and permissively licensed dependencies. However, we also identify gaps in usability: 4.6% of the libraries could not be resolved automatically due to structural mismatches between import names and installable packages, and only two models (out of six) provided installation guidance. While the generated code is technically valid, the lack of contextual support places the burden of manually resolving dependencies on the user. Our findings offer actionable insights for both developers and researchers, and highlight opportunities to improve the reliability and usability of LLM-generated code in the context of software dependencies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。