用大模型自动补全Python类型注解，无需训练即可达到高准确率。
Automated Type Annotation in Python Using Large Language Models
- 基于语法树和类型检查器构建生成-验证-修复循环
- 最优模型达70.5%完全匹配准确率，平均修复少于1次
- 无需微调，适用于其他动态类型语言
Python中的类型注解可提升代码可维护性和错误检测能力。但手动添加注解易出错且耗时。传统自动化方法如静态分析、机器学习和深度学习存在类型词汇有限、行为过度泛化及依赖大规模标注数据等问题。本文探索使用大语言模型（LLM）生成Python类型注解。我们设计了一套生成-检查-修复流水线：LLM在语法树引导下提出注解，Mypy静态类型检查器验证其正确性，错误反馈用于迭代修正。在ManyTypes4Py基准的6000个代码片段上评估了四种LLM：GPT 4oMini、GPT 4.1mini（通用型），以及O3Mini、O4Mini（推理优化型）。结果显示，GPT 4oMini在65.9%的案例中无错误（34.1%不一致），而其他三者一致性均达约88.6%（失败率约11.4%）。在注解质量方面，计算完全匹配与基础类型匹配准确率：GPT 4.1mini与O3Mini表现最佳，分别达到70.5%完全匹配和79.1%基础类型准确率，平均修复次数不足1次。结果表明，通用型与推理优化型LLM无需任务特定微调或额外训练，即可有效生成一致注解，性能可媲美需大量标注数据训练的传统深度学习方法。尽管研究聚焦于Python，该流程可拓展至其他可选类型命令式语言，如Ruby。
原文摘要 · Abstract (English)
Type annotations in Python enhance maintainability and error detection. However, generating these annotations manually is error prone and requires extra effort. Traditional automation approaches like static analysis, machine learning, and deep learning struggle with limited type vocabularies, behavioral over approximation, and reliance on large labeled datasets. In this work, we explore the use of LLMs for generating type annotations in Python. We develop a generate check repair pipeline: the LLM proposes annotations guided by a Concrete Syntax Tree representation, a static type checker (Mypy) verifies them, and any errors are fed back for iterative refinement. We evaluate four LLM variants: GPT 4oMini, GPT 4.1mini (general-purpose), and O3Mini, O4Mini (reasoning optimized), on 6000 code snippets from the ManyTypes4Py benchmark. We first measure the proportion of code snippets annotated by LLMs for which MyPy reported no errors (i.e., consistent results): GPT 4oMini achieved consistency on 65.9% of cases (34.1% inconsistent), while GPT 4.1mini, O3Mini, and O4Mini each reached approximately 88.6% consistency (around 11.4% failures). To measure annotation quality, we then compute exact-match and base-type match accuracies over all 6000 snippets: GPT 4.1mini and O3Mini perform the best, achieving up to 70.5% exact match and 79.1% base type accuracy, requiring under one repair iteration on average. Our results demonstrate that general-purpose and reasoning optimized LLMs, without any task specific fine tuning or additional training can be effective in generating consistent type annotations.They perform competitively with traditional deep learning techniques which require large labeled dataset for training. While our work focuses on Python, the pipeline can be extended to other optionally typed imperative languages like Ruby
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。