SOLARIS提前预计算用户-物品嵌入，让大模型推理更快更省电。
SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling
- 用预测未来请求的思路，提前生成用户-物品嵌入表示。
- 在Meta广告系统上实现0.67%收入指标提升，支撑每日数十亿请求。
- 适合需要高实时性又想用大模型的推荐系统场景。
推荐系统规模扩展定律的进步催生了前所未有的复杂基础模型。尽管性能优越，这些模型的计算开销使实时服务难以实现，常迫使从业者依赖知识蒸馏来换取效率，牺牲服务品质。为解决此问题，我们提出SOLARIS（Speculative Offloading of Latent-bAsed Representation for Inference Scaling），受推测解码启发。SOLARIS通过预测未来可能发生的用户-物品交互对，主动预计算其嵌入表示，并异步生成基础模型表征。该方法将昂贵的基础模型推理从延迟敏感的服务路径中解耦，实现了此前因成本过高而无法在线使用的模型的实时知识传递。部署于支持每日数十亿请求的Meta广告系统，SOLARIS实现了0.67%的收入驱动型核心指标增长，验证了其在大规模应用中的有效性。
原文摘要 · Abstract (English)
Recent advances in recommendation scaling laws have led to foundation models of unprecedented complexity. While these models offer superior performance, their computational demands make real-time serving impractical, often forcing practitioners to rely on knowledge distillation-compromising serving quality for efficiency. To address this challenge, we present SOLARIS (Speculative Offloading of Latent-bAsed Representation for Inference Scaling), a novel framework inspired by speculative decoding. SOLARIS proactively precomputes user-item interaction embeddings by predicting which user-item pairs are likely to appear in future requests, and asynchronously generating their foundation model representations ahead of time. This approach decouples the costly foundation model inference from the latency-critical serving path, enabling real-time knowledge transfer from models previously considered too expensive for online use. Deployed across Meta's advertising system serving billions of daily requests, SOLARIS achieves 0.67% revenue-driving top-line metrics gain, demonstrating its effectiveness at scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。