用大模型自动分析代码数据，提升代码生成质量。
LLM-Aided Customizable Profiling of Code Data Based On Programming Language Concepts
- 用大模型离线学习多种语言的语法语义规则。
- 语法提取准确率达90.33%，语义分类平均77%~80%。
- 可扩展至新语言，适合代码数据质量评估者。
数据概要对机器学习至关重要，能生成描述性统计，支持深层理解及下游任务如数据估值与筛选。本文聚焦代码数据集在大语言模型（code-LLMs）中的概要分析，因数据质量直接影响代码生成与摘要任务。基于编程语言概念表征代码数据，可获得更深入洞察并实现针对性数据筛选。我们提出两阶段方法：（1）离线阶段利用大模型推导并学习跨多种编程语言（包括未见过或低资源语言）的语法与语义概念提取规则；（2）在线确定性阶段应用这些规则进行高效实时分析。该混合方法具备可定制性、可扩展性，且支持多语言。实验表明，该方法在语法提取规则上的平均准确率为90.33%，语义分类准确率在不同语言和概念间平均达80%与77%。
原文摘要 · Abstract (English)
Data profiling is critical in machine learning for generating descriptive statistics, supporting both deeper understanding and downstream tasks like data valuation and curation. This work addresses profiling specifically in the context of code datasets for Large Language Models (code-LLMs), where data quality directly influences tasks such as code generation and summarization. Characterizing code datasets in terms of programming language concepts enables better insights and targeted data curation. Our proposed methodology decomposes code data profiling into two phases: (1) an offline phase where LLMs are leveraged to derive and learn rules for extracting syntactic and semantic concepts across various programming languages, including previously unseen or low-resource languages, and (2) an online deterministic phase applying these derived rules for efficient real-time analysis. This hybrid approach is customizable, extensible to new syntactic and semantic constructs, and scalable to multiple languages. Experimentally, our LLM-aided method achieves a mean accuracy of 90.33% for syntactic extraction rules and semantic classification accuracies averaging 80% and 77% across languages and semantic concepts, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。