用大模型+检索增强，自动推荐代码库迁移方案。
LibRec: Benchmarking Retrieval-Augmented LLMs for Library Migration Recommendations
- 结合大模型与检索增强生成，从提交信息中提取迁移意图
- 在2888条真实迁移记录上验证，准确率显著提升
- 适合做软件维护和代码重构的开发者参考
本文提出LibRec，一种将大语言模型（LLM）与检索增强生成（RAG）技术结合的新框架，用于自动化推荐替代代码库。该框架利用上下文学习从提交信息中提取迁移意图，以提升推荐准确性。为评估其效果，我们构建了LibEval基准，包含从2324个Python仓库中提取的2888条迁移记录，涉及2368个代码库，每条记录包含源-目标库对、迁移意图及意图类型。基于此，我们在框架中评估了十种主流大模型，进行了消融实验分析关键组件贡献，探究不同提示策略的影响，评估了在各类意图类型下的表现，并开展详尽的失败案例分析。
原文摘要 · Abstract (English)
In this paper, we propose LibRec, a novel framework that integrates the capabilities of LLMs with retrieval-augmented generation(RAG) techniques to automate the recommendation of alternative libraries. The framework further employs in-context learning to extract migration intents from commit messages to enhance the accuracy of its recommendations. To evaluate the effectiveness of LibRec, we introduce LibEval, a benchmark designed to assess the performance in the library migration recommendation task. LibEval comprises 2,888 migration records associated with 2,368 libraries extracted from 2,324 Python repositories. Each migration record captures source-target library pairs, along with their corresponding migration intents and intent types. Based on LibEval, we evaluated the effectiveness of ten popular LLMs within our framework, conducted an ablation study to examine the contributions of key components within our framework, explored the impact of various prompt strategies on the framework's performance, assessed its effectiveness across various intent types, and performed detailed failure case analyses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。