提出自动编码器中隐层激活函数的理论选择标准，提升推荐系统精度。
Theoretically informed selection of latent activation in autoencoder based recommender systems
- 基于三个数学性质设计隐层激活函数选择准则
- 发现ReLU/tanh无法同时满足三性质，而sigmoid类更优
- 适合关注推荐系统模型设计效率的研究者
自动编码器可通过将稀疏高维数据压缩为密集低维隐表示，提升推荐系统的准确性和计算效率。然而，由于缺乏理论指导，系统设计仍具挑战性。本文通过理论分析，识别出编码器应具备的三项关键数学性质：(1) 维度压缩，(2) 保持点积比较中的相似性排序，(3) 保留非零向量。结果表明，常见激活函数如ReLU和tanh在通用框架下无法同时满足上述性质。相反，sigmoid类激活函数表现出更优适应性。该理论指导的方法为超参数选择提供了更系统化的路径，显著提升了模型设计效率。
原文摘要 · Abstract (English)
Autoencoders may lend themselves to the design of more accurate and computationally efficient recommender systems by distilling sparse high-dimensional data into dense lower-dimensional latent representations. However, designing these systems remains challenging due to the lack of theoretical guidance. This work addresses this by identifying three key mathematical properties that the encoder in an autoencoder should exhibit to improve recommendation accuracy: (1) dimensionality reduction, (2) preservation of similarity ordering in dot product comparisons, and (3) preservation of non-zero vectors. Through theoretical analysis, we demonstrate that common activation functions, such as ReLU and tanh, cannot fulfill these properties jointly within a generalizable framework. In contrast, sigmoid-like activations emerge as suitable choices for latent activations. This theoretically informed approach offers a more systematic method for hyperparameter selection, enhancing the efficiency of model design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。