用推理节点的闲置算力实时更新推荐模型，不增加同步开销
Near-Zero-Overhead Freshness for Recommendation Systems via Inference-Side Model Updates
- 把低秩适配模块放进推理机，免去参数同步
- 1小时内更新精度比现有方法高0.04%至0.24%
- 适合追求实时推荐的工业级系统
深度学习推荐模型面临新鲜度与准确率的权衡，因嵌入表同步需数分钟，导致推荐质量下降。我们发现推理节点普遍存在CPU空闲（峰值≤20%），且嵌入表梯度具有内在低秩结构。提出LiveUpdate系统，将低秩适配训练器部署在推理节点，消除跨集群同步。解决两大问题：通过奇异值监控动态调整秩，内存开销低于嵌入表的2%；采用NUMA感知调度和硬件保障服务质量，更新推理延迟影响＜20ms。评估显示，相比增量更新基线，更新成本降低2倍，1小时窗口内精度更高。利用闲置推理资源作为新鲜度引擎，实现在线更新，性能优于当前最优增量更新方法。
原文摘要 · Abstract (English)
Deep Learning Recommendation Models (DLRMs) underpin personalized services but face a critical freshness-accuracy tradeoff due to massive parameter synchronization overheads. Production DLRMs deploy decoupled training/inference clusters, where synchronizing petabyte-scale embedding tables (EMTs) causes multi-minute staleness, degrading recommendation quality and revenue. We observe that (1) inference nodes exhibit sustained CPU underutilization (peak <= 20%), and (2) EMT gradients possess intrinsic low-rank structure, enabling compact update representation. We present LiveUpdate, a system that eliminates inter-cluster synchronization by colocating Low-Rank Adaptation (LoRA) trainers within inference nodes. LiveUpdate addresses two core challenges: (1) dynamic rank adaptation via singular value monitoring to constrain memory overhead (<2% of EMTs), and (2) NUMA-aware resource scheduling with hardware-enforced QoS to eliminate update inference contention (P99 latency impact <20ms). Evaluations show LiveUpdate reduces update costs by 2x versus delta-update baselines while achieving higher accuracy within 1-hour windows. By transforming idle inference resources into freshness engines, LiveUpdate delivers online model updates while outperforming state-of-the-art delta-update methods by 0.04% to 0.24% in accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。