arXiv:2509.07928cs.CVcs.AI2025-09

针对消费级显卡优化YOLOv10s推理,实现更快检测速度。

Accelerating Local AI on Consumer GPUs: A Hardware-Aware Dynamic Strategy for YOLOv10s

  • 采用两阶段自适应推理,低分辨率先验快速判断,仅在置信度低时才启用高分辨率
  • 在COCO数据集上比基线快1.85倍,仅损失5.51% mAP
  • 无需修改模型结构,适合部署在笔记本等资源受限设备

随着本地AI日益普及,目标检测器的基准性能与其在消费级硬件上的实际可用性之间存在显著差距。尽管YOLOv10s宣称可实现实时速度,但这些性能通常仅在高性能桌面级GPU上达成。本研究发现,在资源受限系统(如搭载RTX 4060的笔记本)上,性能瓶颈并非来自计算能力,而是系统级限制,通过简单瓶颈测试得以验证。为此,我们提出一种两阶段自适应推理算法,该方法模型无关且无需架构改动。研究聚焦于自适应推理策略,对比分析了架构级早退与分辨率自适应路由的优劣,并在统一评估框架下揭示其权衡。系统采用快速低分辨率前传,仅当检测置信度低时才升级至高分辨率模型。在5000张图像的COCO数据集上,相较PyTorch早退基线,本方法实现1.85倍加速,仅带来5.51%的mAP下降。该工作为在消费级设备上高效部署高性能实时AI提供了可复现的实践蓝图,将优化重心从纯模型设计转向面向硬件的推理策略以最大化吞吐量。

原文摘要 · Abstract (English)

As local AI grows in popularity, there is a critical gap between the benchmark performance of object detectors and their practical viability on consumer-grade hardware. While models like YOLOv10s promise real-time speeds, these metrics are typically achieved on high-power, desktop-class GPUs. This paper reveals that on resource-constrained systems, such as laptops with RTX 4060 GPUs, performance is not compute-bound but is instead dominated by system-level bottlenecks, as illustrated by a simple bottleneck test. To overcome this hardware-level constraint, we introduce a Two-Pass Adaptive Inference algorithm, a model-independent approach that requires no architectural changes. This study mainly focuses on adaptive inference strategies and undertakes a comparative analysis of architectural early-exit and resolution-adaptive routing, highlighting their respective trade-offs within a unified evaluation framework. The system uses a fast, low-resolution pass and only escalates to a high-resolution model pass when detection confidence is low. On a 5000-image COCO dataset, our method achieves a 1.85x speedup over a PyTorch Early-Exit baseline, with a modest mAP loss of 5.51%. This work provides a practical and reproducible blueprint for deploying high-performance, real-time AI on consumer-grade devices by shifting the focus from pure model optimization to hardware-aware inference strategies that maximize throughput.

YOLOv10s自适应推理消费级部署加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。