基于无服务器框架优化词频统计的函数数量，提升处理效率。
Word Frequency Counting Based on Serverless MapReduce
- 在无服务器平台上构建基于MapReduce的词频统计模型。
- 实验表明，增加映射和归约函数数量可降低执行时间并提升效率。
- 为开发者提供最优函数配置参考，适用于大规模数据处理场景。
随着对高性能、高效率计算需求的增长，云计算尤其是无服务器计算近年来逐渐成为研究热点，吸引了大量关注。与此同时，工业界广泛使用的大型数据处理模型MapReduce也得到广泛应用。受函数即服务（Function as a Service）无服务器框架与MapReduce编程模型高并发、强鲁棒性的启发，本文致力于将二者结合，以缩短词频统计任务的执行时间并提升效率。为此，本文在无服务器计算平台上采用基于MapReduce的编程模型，探索特定任务下最优的映射函数（Map）与归约函数（Reduce）数量。针对相同工作负载，大量实验表明，随着映射函数和归约函数数量的增加，执行时间均有所下降，程序整体效率相应提高。本文认为，发现映射与归约函数的最优数量有助于相关开发者和程序员找到最佳解决方案。
原文摘要 · Abstract (English)
With the increasing demand for high-performance and high-efficiency computing, cloud computing, especially serverless computing, has gradually become a research hotspot in recent years, attracting numerous research attention. Meanwhile, MapReduce, which is a popular big data processing model in the industry, has been widely applied in various fields. Inspired by the serverless framework of Function as a Service and the high concurrency and robustness of MapReduce programming model, this paper focus on combining them to reduce the time span and increase the efficiency when executing the word frequency counting task. In this case, the paper use a MapReduce programming model based on a serverless computing platform to figure out the most optimized number of Map functions and Reduce functions for a particular task. For the same amount of workload, extensive experiments show both execution time reduces and the overall efficiency of the program improves at different rates as the number of map functions and reduce functions increases. This paper suppose the discovery of the most optimized number of map and reduce functions can help cooperations and programmers figure out the most optimized solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。