ML代码质量差？这款工具能自动识别76种特定问题。
MLScent A tool for Anti-pattern detection in ML projects
- 基于语法树分析，专为机器学习项目设计
- 在真实项目中准确发现框架特有和数据处理问题
- 适合团队提升模型可复现性与代码可持续性
机器学习代码库因复杂度激增，面临前所未有的代码质量与可持续性挑战。传统代码异味检测工具无法解决影响模型性能、可复现性和可维护性的机器学习特有问题。本文提出 MLScent，一种新型静态分析工具，利用先进的抽象语法树（AST）分析，检测机器学习项目中的反模式与代码异味。该工具涵盖76个独立检测器，覆盖TensorFlow（13个）、PyTorch（12个）、Scikit-learn（9个）、Hugging Face（10个），以及Pandas和NumPy（各8个）。还包含16个通用机器学习异味检测器，以及针对数据预处理和模型训练流程的专项分析。通过量化分类指标与实践者用户研究的定性评估，验证了其在真实项目中对框架特异性反模式、数据处理缺陷及通用机器学习代码异味的高准确性识别能力。
原文摘要 · Abstract (English)
Machine learning (ML) codebases face unprecedented challenges in maintaining code quality and sustainability as their complexity grows exponentially. While traditional code smell detection tools exist, they fail to address ML-specific issues that can significantly impact model performance, reproducibility, and maintainability. This paper introduces MLScent, a novel static analysis tool that leverages sophisticated Abstract Syntax Tree (AST) analysis to detect anti-patterns and code smells specific to ML projects. MLScent implements 76 distinct detectors across major ML frameworks including TensorFlow (13 detectors), PyTorch (12 detectors), Scikit-learn (9 detectors), and Hugging Face (10 detectors), along with data science libraries like Pandas and NumPy (8 detectors each). The tool's architecture also integrates general ML smell detection (16 detectors), and specialized analysis for data preprocessing and model training workflows. Our evaluation demonstrates MLScent's effectiveness through both quantitative classification metrics and qualitative assessment via user studies feedback with ML practitioners. Results show high accuracy in identifying framework-specific anti-patterns, data handling issues, and general ML code smells across real-world projects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。