SwiftEmbed通过静态查表实现毫秒级文本嵌入，适合实时应用。
SwiftEmbed: Ultra-Fast Text Embeddings via Static Token Lookup for Real-Time Applications
- 采用静态嵌入查表+均值池化，避免动态推理
- 单请求延迟仅1.12毫秒，每秒处理5万次请求
- 适合对延迟敏感的实时场景，如重复检测与语义匹配
我们提出SwiftEmbed，一个面向生产环境的静态词元嵌入服务系统，在保持60.6 MTEB平均得分（8个代表性任务）的同时，实现了单文本请求1.12毫秒的p50延迟。该系统基于MinishLab开源的Potion-base-8M蒸馏模型，使用Rust实现，通过静态嵌入查表、均值池化和零拷贝IEEE754二进制序列化，达到每秒5万请求的吞吐量。评估显示其在重复检测上表现优异（90.1% AP），语义相似度相关性达76.1% Spearman。与Sentence-BERT相比，其性能在去重和相似性任务中更优（89–100%），但在分类和复杂检索任务中较低（75%）。在特定领域，其性能为GloVe-840B基线的75%至131%。系统专为子5毫秒延迟至关重要的实时嵌入应用设计，适用于全变压器推理不可行的场景。
原文摘要 · Abstract (English)
We present SwiftEmbed, a production-oriented serving system for static token embeddings that achieves 1.12\,ms p50 latency for single-text requests while maintaining a 60.6 MTEB average score across 8 representative tasks. Built around the open-source Potion-base-8M distilled model from MinishLab and implemented in Rust, the system delivers 50,000 requests per second through static embedding lookup, mean pooling, and zero-copy IEEE754 binary serialization. Evaluation demonstrates exceptional duplicate detection performance (90.1% AP) and strong semantic similarity (76.1% Spearman correlation). Performance relative to Sentence-BERT is task-dependent: robust for deduplication and similarity workloads (89--100%), substantially lower for classification and complex retrieval tasks (75%). Domain-specific performance ranges from 75% to 131% of a GloVe-840B baseline. The system targets real-time embedding applications where sub-5\,ms latency is operationally critical and where full transformer inference is not feasible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。