修复30亿参数模型在苹果芯片上的部署问题,提升代理任务可用性。
Nanbeige4.2-3B on Apple Silicon: Fixing Deployment Bugs and Decreasing Looped Transformer Memory Overhead
- 通过分块预填充策略降低循环变压器的内存开销,提升上下文长度2.7倍
- 修复5个独立缺陷后,模型在标准基准上可完成30%真实代理任务
- 适合关注苹果设备部署、低资源推理优化的研究者与开发者
Nanbeige4.2-3B 是一个基于循环变压器(Looped Transformer, LT)的30亿参数代理模型,通过复用一层结构实现有效深度而无需增加参数。在Apple Silicon(MPS)上评估时,发现五个独立缺陷导致原版检查点无法通过Hugging Face Transformers直接运行(包括一个静默置零的RoPE缓冲区和对已移除缓存API的调用)。此外,由于LT的层复用策略使峰值注意力内存翻倍,仅修复缺陷仍不足以支持代理任务。为此提出分块预填充策略,将32 GiB共享内存下的允许上下文宽度扩展2.7倍。即便如此,仍需修复系统提示与MPS原生内存问题才能可靠评估。在MCPMark子集上,调试后的模型可完成30%的真实代理任务(原为0%),在BFCL单工具调用中接近完美,但多数多工具任务仍失败。相关补丁检查点、系统提示优化器与评估工具已开源。
原文摘要 · Abstract (English)
Nanbeige4.2-3B is a 3B-parameter agentic model built around a Looped Transformer (LT) that reuses one stack of layers for a second forward pass, adding effective depth without additional parameters. Evaluated on Apple Silicon (MPS), we identify five independent bugs which prevent the released checkpoint from running via Hugging Face transformers out of the box (including a silently-zeroed RoPE buffer and calls to removed transformers cache APIs). Furthermore, we show that fixing these bugs is still not sufficient for agentic tasks, due to the LT's layer-reuse strategy (which effectively doubles peak attention memory) used to achieve parameter efficiency. We thus introduce a chunked-prefill strategy which alleviates the incurred memory-capacity penalty, extending allowable context width by $2.7 \times$ on 32~GiB shared memory. However, even with the reduced memory overhead, we show that patches are required to render Nanbeige4.2-3B usable; resolving both system prompt and MPS-native memory bugs finally allows reliable evaluation on standard MCP and tool-calling benchmarks. On a subset of MCPMark, the debugged model completes up to 30\% of real agentic tasks (up from the original's 0\%), while, on BFCL, it is near-perfect at single tool calls (yet fails the majority of multi-tool tests). We release the patched checkpoint, system prompt optimizer, and evaluation harnesses at https://github.com/johnhalloran321/Nanbeige4.2-3B-mps-fix.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。