arXiv:2512.14141cs.CVcs.AI2025-12

首个专用于检测视觉模型性能反模式的基准数据集,提升自动化分析效率。

TorchTraceAP: A New Benchmark Dataset for Detecting Performance Anti-Patterns in Computer Vision Models

  • 构建600+跨平台视觉模型的PyTorch执行轨迹数据集
  • 轻量模型定位反模式段,大模型精准分类反馈,准确率显著优于传统方法
  • 适合研究模型优化、系统调优及自动化诊断的开发者使用

识别和解决机器学习模型中的性能反模式对高效训练与推理至关重要,但通常需要涵盖系统基础设施、模型结构和内核开发的深厚专业知识。大型科技公司依赖专职的ML基础设施工程师分析Torch追踪日志和基准测试,而这类高成本流程对普通计算机视觉研究者难以企及。其中,从长序列执行轨迹中精确定位问题段落是最耗时的任务,且当前的机器学习模型(包括大语言模型)难以有效自动化。本文提出首个专门用于评估和改进模型检测追踪中反模式能力的基准数据集。数据集包含超过600个来自多种视觉模型(分类、检测、分割、生成)的PyTorch追踪日志,覆盖多个硬件平台。我们还提出一种新型迭代方法:先用轻量级模型检测含反模式的追踪段,再由大语言模型进行细粒度分类与针对性反馈。实验表明,该方法在检测反模式区域方面显著优于无监督聚类与基于规则的统计技术,同时有效弥补了大模型上下文长度限制与推理效率不足的问题。

原文摘要 · Abstract (English)

Identifying and addressing performance anti-patterns in machine learning (ML) models is critical for efficient training and inference, but it typically demands deep expertise spanning system infrastructure, ML models and kernel development. While large tech companies rely on dedicated ML infrastructure engineers to analyze torch traces and benchmarks, such resource-intensive workflows are largely inaccessible to computer vision researchers in general. Among the challenges, pinpointing problematic trace segments within lengthy execution traces remains the most time-consuming task, and is difficult to automate with current ML models, including LLMs. In this work, we present the first benchmark dataset specifically designed to evaluate and improve ML models' ability to detect anti patterns in traces. Our dataset contains over 600 PyTorch traces from diverse computer vision models classification, detection, segmentation, and generation collected across multiple hardware platforms. We also propose a novel iterative approach: a lightweight ML model first detects trace segments with anti patterns, followed by a large language model (LLM) for fine grained classification and targeted feedback. Experimental results demonstrate that our method significantly outperforms unsupervised clustering and rule based statistical techniques for detecting anti pattern regions. Our method also effectively compensates LLM's limited context length and reasoning inefficiencies.

性能优化PyTorch反模式检测AI工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。