用统计模型重构嵌入，大幅减少参数量并防过拟合。
MMbeddings: Parameter-Efficient, Low-Overfitting Probabilistic Embeddings Inspired by Nonlinear Mixed Models
- 将嵌入视为随机效应，在变分自编码器中建模
- 参数量与类别数无关，显著降低过拟合风险
- 适合高基数场景，如推荐系统和表格回归
我们提出MMbeddings，一种基于非线性混合模型的概率嵌入方法，将传统分类嵌入重新诠释为变分自编码器中的潜在随机效应。该方法将参数量从传统的类别数×嵌入维度（高基数下不可行）降低为与类别数无关的更小值，主要由编码器架构决定。在模拟数据和真实数据上的大量实验表明，包括协同过滤和表格回归任务，使用多种网络结构时，MMbeddings均持续优于传统嵌入方法，展现了其在多样化机器学习应用中的潜力。
原文摘要 · Abstract (English)
We present MMbeddings, a probabilistic embedding approach that reinterprets categorical embeddings through the lens of nonlinear mixed models, effectively bridging classical statistical theory with modern deep learning. By treating embeddings as latent random effects within a variational autoencoder framework, our method substantially decreases the number of parameters -- from the conventional embedding approach of cardinality $\times$ embedding dimension, which quickly becomes infeasible with large cardinalities, to a significantly smaller, cardinality-independent number determined primarily by the encoder architecture. This reduction dramatically mitigates overfitting and computational burden in high-cardinality settings. Extensive experiments on simulated and real datasets, encompassing collaborative filtering and tabular regression tasks using varied architectures, demonstrate that MMbeddings consistently outperforms traditional embeddings, underscoring its potential across diverse machine learning applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。