用大模型+IDE+语义检索,自动完成代码重构推荐与执行
Leveraging LLMs, IDEs, and Semantic Embeddings for Automated Move Method Refactoring
- 结合大模型、IDE静态分析与自检机制,过滤80%的错误建议
- 在开源项目上召回率提升2.4倍,比现有方法更准确
- 适合需要高效代码重构的开发者,尤其擅长全局性重构
MOVEMETHOD是关键的代码重构操作。尽管已有众多研究工具推荐移动方法及目标位置,但其建议与专家实践不一致。我们发现大语言模型虽能给出专家级建议,但存在高达80%的幻觉。为此,我们提出首个端到端自动化MOVEMETHOD重构助手MM-assist,通过IDE静态分析过滤幻觉,设计自洽性检查与建议排序流程,并采用重构感知的检索增强生成(RAG)解决大模型上下文限制。在多方法评估中,相较于现有最优方法,我们在广泛使用的基准上召回率@1和@3提升1.7倍;在包含210个近期开源重构的语料上,召回率至少提升2.4倍。30名资深开发者使用一周后,82.8%的建议获得正面评价,表明该工具兼具有效性与实用性。
原文摘要 · Abstract (English)
MOVEMETHOD is a hallmark refactoring. Despite a plethora of research tools that recommend which methods to move and where, these recommendations do not align with how expert developers perform MOVEMETHOD. Given the extensive training of Large Language Models and their reliance upon naturalness of code, they should expertly recommend which methods are misplaced in a given class and which classes are better hosts. Our formative study of 2016 LLM recommendations revealed that LLMs give expert suggestions, yet they are unreliable: up to 80% of the suggestions are hallucinations. We introduce the first LLM fully powered assistant for MOVEMETHOD refactoring that automates its whole end-to-end lifecycle, from recommendation to execution. We designed novel solutions that automatically filter LLM hallucinations using static analysis from IDEs and a novel workflow that requires LLMs to be self-consistent, critique, and rank refactoring suggestions. As MOVEMETHOD refactoring requires global, projectlevel reasoning, we solved the limited context size of LLMs by employing refactoring-aware retrieval augment generation (RAG). Our approach, MM-assist, synergistically combines the strengths of the LLM, IDE, static analysis, and semantic relevance. In our thorough, multi-methodology empirical evaluation, we compare MM-assist with the previous state-of-the-art approaches. MM-assist significantly outperforms them: (i) on a benchmark widely used by other researchers, our Recall@1 and Recall@3 show a 1.7x improvement; (ii) on a corpus of 210 recent refactorings from Open-source software, our Recall rates improve by at least 2.4x. Lastly, we conducted a user study with 30 experienced participants who used MM-assist to refactor their own code for one week. They rated 82.8% of MM-assist recommendations positively. This shows that MM-assist is both effective and useful.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。