arXiv:2505.00816cs.SEcs.LG2025-05被引 1

量化模型能显著提升效率,仅轻微影响准确率。

Aggregating empirical evidence from data strategy studies: a case on model quantization

  • 用结构化合成法整合六项实证研究证据
  • 量化使存储、延迟、能耗降低,准确率小幅下降
  • 适合关注模型部署优化的研究者与工程师

背景:随着实证软件工程的发展,越来越多研究采用数据策略——即通过分析模型、源代码或系统日志等数字产物,而非依赖人类受试者。此类研究结果的整合带来新的方法论挑战。目标:评估模型量化对深度学习系统正确性与资源效率的影响,并探讨基于数据策略的实证研究证据聚合的方法学意义。方法:对六项实证评估模型量化的原始研究进行研究综合,采用结构化合成法(SSM)整合定性与定量证据,通过图示建模方式。共提取并聚合19个证据模型。结果:聚合证据表明,模型量化对正确性指标有微弱负面影响,但一致提升资源效率指标,包括存储大小、推理延迟和GPU能耗,这一权衡在多数深度学习部署场景中可接受。不同量化技术间的证据仍呈碎片化状态,凸显需针对每种技术开展更聚焦的实证研究。结论:模型量化在资源受限环境中具有显著效率优势,仅伴随轻微正确性损失,是一种合适的优化策略。本研究也验证了使用SSM整合数据策略类研究发现的可行性。

原文摘要 · Abstract (English)

Background: As empirical software engineering evolves, more studies adopt data strategies$-$approaches that investigate digital artifacts such as models, source code, or system logs rather than relying on human subjects. Synthesizing results from such studies introduces new methodological challenges. Aims: This study assesses the effects of model quantization on correctness and resource efficiency in deep learning (DL) systems. Additionally, it explores the methodological implications of aggregating evidence from empirical studies that adopt data strategies. Method: We conducted a research synthesis of six primary studies that empirically evaluate model quantization. We applied the Structured Synthesis Method (SSM) to aggregate the findings, which combines qualitative and quantitative evidence through diagrammatic modeling. A total of 19 evidence models were extracted and aggregated. Results: The aggregated evidence indicates that model quantization weakly negatively affects correctness metrics while consistently improving resource efficiency metrics, including storage size, inference latency, and GPU energy consumption$-$a manageable trade-off for many DL deployment contexts. Evidence across quantization techniques remains fragmented, underscoring the need for more focused empirical studies per technique. Conclusions: Model quantization offers substantial efficiency benefits with minor trade-offs in correctness, making it a suitable optimization strategy for resource-constrained environments. This study also demonstrates the feasibility of using SSM to synthesize findings from data strategy-based research.

模型量化实证研究效率优化数据策略

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。