提出新性能定律与近似熵，提升推荐模型在不同规模下的预测精度。
Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy
- 基于变压器模型拟合指标，建立推荐系统性能定律
- 用近似熵衡量数据质量，显著优于仅看数据量的方法
- 适用于各类模型和数据规模，助你选最优配置
缩放定律已成为理解模型性能随规模增长的有力框架，为优化计算资源提供洞见。在序列推荐（SR）领域，该框架可帮助应对模型扩展性挑战。然而，推荐系统中的结构与协同问题阻碍了缩放定律的直接应用。为此，本文提出面向SR模型的性能定律，旨在理论分析并建模模型性能与数据质量的关系。首先，将HR和NDCG指标拟合至基于Transformer的SR模型；其次，提出近似熵（ApEn）用于评估数据质量，相比传统数量指标更具细致性。该方法可在多种数据规模和模型尺寸下实现精准预测，在大规模SR模型中表现出强相关性，并为任意模型配置提供性能最优化指导。
原文摘要 · Abstract (English)
Scaling Laws have emerged as a powerful framework for understanding how model performance evolves as they increase in size, providing valuable insights for optimizing computational resources. In the realm of Sequential Recommendation (SR), which is pivotal for predicting users' sequential preferences, these laws offer a lens through which to address the challenges posed by the scalability of SR models. However, the presence of structural and collaborative issues in recommender systems prevents the direct application of the Scaling Law (SL) in these systems. In response, we introduce the Performance Law for SR models, which aims to theoretically investigate and model the relationship between model performance and data quality. Specifically, we first fit the HR and NDCG metrics to transformer-based SR models. Subsequently, we propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics. Our method enables accurate predictions across various dataset scales and model sizes, demonstrating a strong correlation in large SR models and offering insights into achieving optimal performance for any given model configuration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。