在苹果芯片上实现本地全模态搜索，文件全程不离开设备
omni-macos: On-Device Omni-Modal Search on Apple Silicon
- 将文本、代码、图像等统一编码并本地存储，无需云端
- 仅重编码修改部分，支持实时搜索响应
- 适配不同配置的Mac，内存预算自动分配
一个能将文本、代码、文档、图像、音频和视频嵌入同一表征空间的搜索引擎，通常依赖服务器运行编码器与索引。我们提出 omni-macos，将其编码器、索引和存储全部部署在已持有文件的 Mac 上，确保任何文件、查询或向量均不离开设备。系统采用后台索引机制与用户可设内存预算：仅对编辑变更的片段重新编码，用户输入时以小单位交由GPU处理，通过一比特索引副本快速响应，并进行精确重评分；同时将该预算动态传递给统一内存分配器。我们在五台覆盖八倍加速器宽度、三十二倍内存差异的Mac上进行了测试，每台均对自身已有文件进行索引。
原文摘要 · Abstract (English)
A search engine that embeds text, code, documents, images, audio and video into the same representation space has to run its encoder and keep its index somewhere, and almost every component built for the purpose assumes a server. We present omni-macos, which runs its encoder, index and store on the Mac that already holds the files, so no indexed file, no typed query and no vector ever leaves the machine. It keeps a background indexer and an interactive search box inside one memory budget the user sets: it re-encodes only the chunks an edit changes, hands the GPU smaller units while the user is typing, answers queries from a one-bit replica of the index with exact rescoring, and propagates that budget to the allocators that draw on unified memory. We measure on five Macs spanning an eightfold range of accelerator width and a thirty-twofold range of memory, each indexing the files it already holds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。