用知识图谱和多准则决策,让选Python包更透明可靠。
Empirical Evaluation of AI-Assisted Software Package Selection: A Knowledge Graph Approach
- 构建包含依赖、漏洞、用户反馈的软件知识图谱,支持多维度评估。
- 在79.8万份代码上测试,推荐质量优于纯AI生成方案。
- 适合关注安全、可维护性的开发者和团队使用。
在Python等开源生态中,选择第三方软件包面临选项繁多且缺乏透明比较证据的问题。生成式AI工具常忽略依赖关系,过度依赖流行度,缺乏可复现性,带来长期风险。本文将包选择问题建模为多准则决策(MCDM),提出数据驱动的评估框架。通过自动化管道持续采集GitHub、PyPI、Stack Overflow的元数据、使用趋势、漏洞信息与开发者情绪,并构建表示包、领域特征与质量属性关系的知识图谱。该框架实现为PySelect系统,利用大模型理解用户意图并查询图谱推荐合适包。基于16,887个GitHub仓库中的798,669份Python脚本及基于技术接受模型的用户研究验证,结果表明数据提取精度高,推荐质量优于生成式AI基线,用户对系统有用性和易用性评价积极。本工作提供可扩展、可解释、可复现的证据驱动包选择方案。
原文摘要 · Abstract (English)
Selecting third-party software packages in open-source ecosystems like Python is challenging due to the large number of alternatives and limited transparent evidence for comparison. Generative AI tools are increasingly used in development workflows, but their suggestions often overlook dependency evaluation, emphasize popularity over suitability, and lack reproducibility. This creates risks for projects that require transparency, long-term reliability, maintainability, and informed architectural decisions. This study formulates software package selection as a Multi-Criteria Decision-Making (MCDM) problem and proposes a data-driven framework for technology evaluation. Automated data pipelines continuously collect and integrate software metadata, usage trends, vulnerability information, and developer sentiment from GitHub, PyPI, and Stack Overflow. These data are structured into a decision model representing relationships among packages, domain features, and quality attributes. The framework is implemented in PySelect, a decision support system that uses large language models to interpret user intent and query the model to identify contextually appropriate packages. The approach is evaluated using 798,669 Python scripts from 16,887 GitHub repositories and a user study based on the Technology Acceptance Model. Results show high data extraction precision, improved recommendation quality over generative AI baselines, and positive user evaluations of usefulness and ease of use. This work introduces a scalable, interpretable, and reproducible framework that supports evidence-based software selection using MCDM principles, empirical data, and AI-assisted intent modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。