arXiv:2605.17172cs.LGcs.AI2026-05被引 2

让本地AI在不依赖云端的情况下,逼近大模型性能。

OpenJarvis: Personal AI, On Personal Devices

论文配图:OpenJarvis: Personal AI, On Personal Devices
图 1 · 摘自论文原文
  • 将个人AI系统拆解为五个可独立优化的组件
  • 本地模型性能接近云端,平均差距仅3.2个百分点
  • 显著降低使用成本和延迟,适合注重隐私的用户

个人AI系统如OpenClaw和Hermes Agent正成为日常工作的核心,但几乎所有的请求(常涉及敏感本地数据)仍需发送至云端的前沿模型处理。将前沿模型替换为本地模型效果不佳:用Qwen3.5-9B替代Claude Opus 4.6,导致PinchBench和GAIA等任务准确率下降25-39个百分点。现有系统将提示词、工具描述、记忆配置与运行时设置绑定于特定云端模型,仅提示词可调优,当前最优提示优化器也仅缩小5个百分点的差距。为此,我们提出OpenJarvis,一种解耦的个人AI架构,以五类独立可编辑的原语(智能、引擎、代理、工具与记忆、学习)构成类型化规范,支持端到端可优化与可度量。为在不牺牲本地模型优势的前提下缩小本地-云端差距,OpenJarvis引入LLM引导的规范搜索:前沿云模型在搜索阶段提出修改建议,仅接受不退化的调整,最终规范完全在设备端运行。实验表明,经此优化的本地规范在8个基准中的4个上达到或超过云端表现,平均差距仅3.2个百分点,边际API成本降低约800倍,端到端延迟减少4倍。

原文摘要 · Abstract (English)

Personal AI stacks, like OpenClaw and Hermes Agent, are becoming central to daily work, yet they route nearly every query (often over sensitive local data) to cloud-hosted frontier models. Replacing frontier models with local models inside existing stacks does not work: swapping Claude Opus 4.6 for Qwen3.5-9B drops accuracy by 25-39 pp across personal AI tasks like PinchBench and GAIA. Existing stacks bundle agentic prompts, tool descriptions, memory configuration, and runtime settings around a specific cloud model. Only the prompts can be tuned, and state-of-the-art prompt optimizers close just 5 pp of the local-cloud gap on their own. This motivates a decomposed personal AI stack: one that exposes individual primitives which can be optimized individually or jointly to close the local-cloud gap. We present OpenJarvis, an architecture that represents a personal AI system as a typed spec over five primitives: Intelligence, Engine, Agents, Tools & Memory, and Learning. Each primitive is an independently editable field, making the stack end-to-end optimizable and measurable against accuracy, cost, and latency. Towards closing the local-cloud gap without surrendering local-model properties, OpenJarvis introduces LLM-guided spec search, a local-cloud collaboration in which frontier cloud models propose edits across the spec at search time, only non-regressing edits are accepted, and the resulting spec runs entirely on-device at inference time. With LLM-guided spec search, on-device specs match or exceed cloud accuracy on 4 of 8 benchmarks and land within 3.2 pp of the best cloud baseline on average. They also reduce marginal API cost by ~800x and end-to-end latency by 4x.

个人AI本地部署模型优化隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。