在苹果芯片上实现波兰语大模型跨家族快速推理，提升生成效率。
Cross-Family Speculative Decoding for Polish Language Models on Apple~Silicon: An Empirical Evaluation of Bielik~11B with UAG-Extended MLX-LM
- 用通用辅助生成技术打通不同分词器的模型协作，支持苹果硅芯片推理。
- 上下文感知翻译使生成接受率提升，但专用小模型效果不如通用模型。
- 结构化文本可提速1.7倍，但复杂指令下反而变慢，受内存带宽限制。
推测解码通过小模型预提候选项加速大模型推理，但在跨家族、分词器不匹配及消费级统一内存环境下尚未充分研究。本文扩展MLX-LM框架，引入通用辅助生成（UAG），实现在苹果硅芯片上的跨分词器推测解码。以基于Mistral的Bielik 11B-Instruct为靶模型，搭配Bielik 1.5B（Qwen基，自定义分词器）、Qwen2.5-1.5B和Llama 3.2-1B三个小模型，在Wikipedia、pl_alpaca与合成数据集上测试，候选长度k取{2,4,6}，对比朴素与上下文感知的令牌转换。结果表明：(1) 上下文感知转换在所有配置中均提升接受率；(2) 波兰语专用的Bielik 1.5B接受率低于通用型的Qwen2.5和Llama 3.2；(3) 苹果硅芯片上吞吐量依赖内容，结构化文本达1.7倍加速，但多样指令下失败；(4) 统一内存下验证开销未如理论预期地摊薄，因双模型均受内存带宽制约，串行草案代价高过批量验证。提出硬件感知加速公式，并刻画苹果硅上跨家族推测解码条件。本研究首次系统评估波兰语大模型跨家族推测解码，也是首个基于UAG在统一内存架构上的实证研究。
原文摘要 · Abstract (English)
Speculative decoding accelerates LLM inference by using a small draft model to propose k candidate tokens for a target model to verify. While effective for same-tokenizer pairs on high-bandwidth GPUs, its applicability to cross-family pairs with mismatched tokenizers and consumer-grade unified memory remains underexplored. We extend the MLX-LM framework with Universal Assisted Generation (UAG) to enable cross-tokenizer speculative decoding on Apple Silicon. We evaluate Bielik 11B-Instruct (Mistral-based) as the target model, paired with three draft models: Bielik 1.5B (Qwen-based with custom tokenizer), Qwen2.5-1.5B, and Llama 3.2-1B. Experiments on three Polish-language datasets (Wikipedia, pl_alpaca, synthetic) use draft lengths k in {2, 4, 6} to compare naive and context-aware token translation. Results show: (1) context-aware translation consistently improves acceptance rates across all configurations; (2) the Polish-specialized Bielik 1.5B achieves lower acceptance than general-purpose Qwen2.5 and Llama 3.2 drafters; (3) throughput on Apple Silicon is content-dependent, reaching 1.7x speedup for structured text but failing for varied instructions; and (4) verification cost on unified memory does not amortize as theory predicts because both models are memory-bandwidth bound, making sequential drafting expensive relative to batched verification. We propose a hardware-aware speedup formula and characterize conditions for cross-family speculative decoding on Apple Silicon. This is the first systematic evaluation of cross-family speculative decoding for Polish LLMs and the first empirical study of UAG-based decoding on unified memory architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。