对比12种JPEG解码器在不同CPU上的速度,发现社交媒体图片解码快40%。
Choosing a JPEG Decoder for PyTorch DataLoaders: Workload-Specific Throughput on Four CPUs
- 固定8个工作线程,测试12种Python解码器在4种CPU上的性能
- 处理社交平台图片时,部分解码器比Pillow快39%-40%
- 推荐用代表性图片测试,以选择最佳解码器
通过固定PyTorch DataLoader为8个工作线程,对比12种Python JPEG解码器在4个Google Cloud 16-vCPU平台上的性能。使用来自Forchheim图像数据库的两个工作负载:324张大型相机原图,以及1,668张原图与社交平台处理后的图片混合数据。每个解码器-处理器-工作负载组合重复5次独立实验。所有JPEG数据驻留内存,每次解码返回完整的三通道RGB数组。结果表明,在相机原图上,各解码器相对于Pillow的几何平均加速比为1.00–1.09倍;而在混合工作负载中,simplejpeg、imagecodecs和turbojpeg可达到1.39–1.40倍的加速。具体领先者随CPU变化,因此建议在代表性图片上进行比较,同时以八线程结果作为紧凑起点。完整的工作线程曲线与绝对吞吐量表格见附录。
原文摘要 · Abstract (English)
A JPEG decoder benchmark can combine worker counts, CPUs, and datasets in one large result matrix. We simplify that comparison by fixing a PyTorch DataLoader at eight workers and asking one question: how much faster is each decoder than Pillow on the same CPU and JPEG workload? We benchmark 12 Python decoders on four 16-vCPU Google Cloud platforms and two workloads from the Forchheim Image Database. The first contains 324 large camera originals. The second contains 1,668 originals and copies processed by social media services. Every decoder-CPU-workload setting has five isolated repetitions. JPEG bytes reside in memory, and each timed decode returns a complete three-channel RGB array. The workload changes the practical result. On camera originals, the geometric mean across CPUs ranges from 1.00 to 1.09 times Pillow. On the mixed workload, simplejpeg, imagecodecs, and turbojpeg reach 1.39 - 1.40 Pillow. The exact leader still changes by CPU. We therefore recommend comparing decoders on representative JPEGs, while using the fixed eight-worker results as a compact starting point. Full worker curves and absolute throughput tables are in the appendix.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。