arXiv:2507.13575cs.LGcs.AI2025-07被引 84

苹果推出两款新语言模型,支持多语言图文理解与工具调用,兼顾性能与隐私。

Apple Intelligence Foundation Language Models: Tech Report 2025

  • 采用30亿参数的设备端模型,通过共享缓存与2比特量化训练优化运行效率。
  • 服务器模型基于并行轨迹专家架构,在私有云平台实现高质量生成且成本可控。
  • 提供易用的Swift开发框架,适合想快速集成AI功能的开发者使用。

我们介绍了两款多语言、多模态的基础语言模型,用于支撑苹果设备与服务中的Apple Intelligence功能:一是在苹果芯片上优化的30亿参数设备端模型,通过键值缓存共享和2比特量化感知训练实现高效运行;二是基于新型并行轨迹专家(PT-MoE)Transformer的可扩展服务器模型,结合轨迹并行、专家混合稀疏计算与交错全局-局部注意力,在苹果私有云计算平台上实现高质量输出且成本具有竞争力。两个模型均在大规模多语言多模态数据集上训练,数据来源包括负责任的网络爬取、授权语料库及高质量合成数据,并通过新异步平台进行监督微调与强化学习优化。模型新增多种语言支持,具备图像理解与工具调用能力。在公开基准测试与人工评估中,两者表现均达到或超越同等规模开源基线。新推出的以Swift为中心的基础模型框架,支持引导生成、受限工具调用与LoRA适配器微调,开发者仅需少量代码即可集成。最新进展建立在负责任AI理念之上,包含内容过滤、本地化评估等保障措施,并通过私有云计算等创新技术确保用户隐私安全。

原文摘要 · Abstract (English)

We introduce two multilingual, multimodal foundation language models that power Apple Intelligence features across Apple devices and services: i a 3B-parameter on-device model optimized for Apple silicon through architectural innovations such as KV-cache sharing and 2-bit quantization-aware training; and ii a scalable server model built on a novel Parallel-Track Mixture-of-Experts PT-MoE transformer that combines track parallelism, mixture-of-experts sparse computation, and interleaved global-local attention to deliver high quality with competitive cost on Apple's Private Cloud Compute platform. Both models are trained on large-scale multilingual and multimodal datasets sourced via responsible web crawling, licensed corpora, and high-quality synthetic data, then further refined with supervised fine-tuning and reinforcement learning on a new asynchronous platform. The resulting models support several additional languages while understanding images and executing tool calls. In public benchmarks and human evaluations, both the server model and the on-device model match or surpass comparably sized open baselines. A new Swift-centric Foundation Models framework exposes guided generation, constrained tool calling, and LoRA adapter fine-tuning, allowing developers to integrate these capabilities with a few lines of code. The latest advancements in Apple Intelligence models are grounded in our Responsible AI approach with safeguards like content filtering and locale-specific evaluation, as well as our commitment to protecting our users' privacy with innovations like Private Cloud Compute.

大模型苹果隐私保护多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。