小模型的稳定性不靠参数大小，关键在工程设计
It's Not the Size: Harness Design Determines Operational Stability in Small Language Models
- 用四阶段流水线提升小模型任务成功率
- 流水线使成功率达95.2%，验证捕获率62.5%
- 简单包装反而降低表现，暴露结构崩溃问题
本文实验分析了小型语言模型（SLMs，2-3B参数）中牵制工程对其运行性能的影响。在三个模型（Gemma4 E2B、Qwen3.5:2B、LLaMA 3.2 3B）上，对24个任务测试三种牵制方式：仅模型（原始提示）、极简外壳（包装标签）和四阶段流水线（规划→执行→验证→恢复）。流水线在Gemma4 E2B上实现任务成功率（TSR）0.952，有效任务成功率（VTSR）1.000（T1-T5，21项任务）。在两个模型中观察到非单调现象：极简外壳的TSR低于仅模型。在LLaMA 3.2 3B中，仅模型因7次格式违规导致TSR降至0.429，揭示结构崩溃：复杂格式要求下模型放弃JSON结构。消融实验显示，规划与恢复各贡献约24.7%的总增益。所有流水线运行的验证捕获率（VCR）为0.625。
原文摘要 · Abstract (English)
This paper experimentally analyzes how the level of harness engineering affects the operational performance of small language models (SLMs, 2-3B parameters). Three harness conditions - model-only (raw prompt), minimal-shell (wrapper tags), and a 4-stage pipeline (plan->execute->verify->recover) - are applied to three models (Gemma4 E2B, Qwen3.5:2B, LLaMA 3.2 3B) across 24 tasks, comparing Task Success Rate (TSR) and Valid TSR (VTSR). The pipeline harness achieves TSR=0.952 and VTSR=1.000 on Gemma4 E2B (T1-T5, 21 tasks). A non-monotonic phenomenon - minimal-shell TSR < model-only TSR - is observed in two models. In LLaMA 3.2 3B model-only, seven format violations yield TSR=0.429, revealing scaffold collapse: the model abandons JSON structure under complex format requirements without harness support. Ablation shows planning and recovery each contribute approximately 24.7% of total gain. VCR (Verification Catch Rate)=0.625 across all pipeline runs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。