专为代码标识符设计的词性标注工具，提升开发语言结构识别准确率。
SCALAR: A Part-of-speech Tagger for Identifiers
- 基于梯度提升分类器，结合人工标注语料训练专用词性标注模型。
- 在标识符标注任务上优于旧版标签器和通用词性标注工具。
- 适合代码分析、自动化重构等需要理解变量命名规律的场景。
本文提出 Source Code Analysis and Lexical Annotation Runtime (SCALAR)，一种专门用于将源代码标识符名称映射到其对应词性标记序列（语法模式）的工具。SCALAR 的内部模型使用 scikit-learn 的 GradientBoostingClassifier，并结合人工构建的标识符名称及其语法模式语料库进行训练，使其能够识别开发者用于各类标识符（如函数名、变量名等）的独特自然语言结构。通过与前代标签器及现代现成词性标注工具对比，验证了 SCALAR 在标识符标注任务上的优越性。代码已开源于 Github。
原文摘要 · Abstract (English)
The paper presents the Source Code Analysis and Lexical Annotation Runtime (SCALAR), a tool specialized for mapping (annotating) source code identifier names to their corresponding part-of-speech tag sequence (grammar pattern). SCALAR's internal model is trained using scikit-learn's GradientBoostingClassifier in conjunction with a manually-curated oracle of identifier names and their grammar patterns. This specializes the tagger to recognize the unique structure of the natural language used by developers to create all types of identifiers (e.g., function names, variable names etc.). SCALAR's output is compared with a previous version of the tagger, as well as a modern off-the-shelf part-of-speech tagger to show how it improves upon other taggers' output for annotating identifiers. The code is available on Github
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。