提出异步推理框架,让推荐系统预排阶段更快更省资源。
AIF: Asynchronous Inference Framework for Cost-Effective Pre-Ranking
- 将用户和物品的独立计算提前并行处理,打破原有串行瓶颈。
- 在淘宝广告系统中实现延迟降低35%、资源节省40%以上。
- 适合大规模工业推荐系统优化,尤其关注效率与成本的场景。
在工业级推荐系统中,基于深度神经网络的预排模型通常采用串行执行框架:只有在上游召回阶段收到候选集后,才触发特征获取和模型前向计算。这种设计存在固有瓶颈,包括相同用户/物品的重复计算以及严格串行操作带来的延迟增加,共同制约了模型能力与系统效率。为此,我们提出异步推理框架(AIF),一种低成本计算架构,将不依赖交互的组件(单个用户或物品内)从实时预测中解耦。AIF通过在召回阶段并行完成用户侧计算,并以近线方式处理物品侧计算,使交互无关组件仅需计算一次且在实时预排前完成。结果显著提升计算效率,降低延迟,释放资源以大幅扩展交互无关组件的特征集与模型结构。此外,我们在AIF框架内设计模型,对交互相关组件采用近似方法进行在线实时预测。通过框架与模型协同设计,方案在几乎不增加计算与延迟成本的前提下实现显著性能提升,已在淘宝展示广告系统成功部署。
原文摘要 · Abstract (English)
In industrial recommendation systems, pre-ranking models based on deep neural networks (DNNs) commonly adopt a sequential execution framework: feature fetching and model forward computation are triggered only after receiving candidates from the upstream retrieval stage. This design introduces inherent bottlenecks, including redundant computations of identical users/items and increased latency due to strictly sequential operations, which jointly constrain the model's capacity and system efficiency. To address these limitations, we propose the Asynchronous Inference Framework (AIF), a cost-effective computational architecture that decouples interaction-independent components, those operating within a single user or item, from real-time prediction. AIF reorganizes the model inference process by performing user-side computations in parallel with the retrieval stage and conducting item-side computations in a nearline manner. This means that interaction-independent components are calculated just once and completed before the real-time prediction phase of the pre-ranking stage. As a result, AIF enhances computational efficiency and reduces latency, freeing up resources to significantly improve the feature set and model architecture of interaction-independent components. Moreover, we delve into model design within the AIF framework, employing approximated methods for interaction-dependent components in online real-time predictions. By co-designing both the framework and the model, our solution achieves notable performance gains without significantly increasing computational and latency costs. This has enabled the successful deployment of AIF in the Taobao display advertising system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。