7B模型在免费显卡上用适配器接力微调,但合成数据问题导致效果反而变差。
Fine-Tuning a 7B Advisor on Free-Tier GPUs: An Adapter-Handoff Recipe and a Synthetic-Data Reliability Caution
- 仅传输小体积适配器(4190万参数),跨两台免费显卡接力完成微调。
- 微调后模型在合成数据相似度上提升0.063,但实际建议质量反而下降。
- 适合关注低成本微调与合成数据可信度的研究者或实践者。
在资源受限环境下微调7B语言模型用于专业建议颇具吸引力,但多轮训练常超出免费级GPU(如Kaggle、Colab)的时长限制。我们提出一种实用方案:通过Unsloth实现三轮QLoRA微调(Mistral-7B-Instruct-v0.3,4-bit NF4,LoRA rank 16),仅保存并迁移小型适配器(4190万参数),在两台16GB免费显卡(Tesla P100后接T4)间接力完成。结果表明,仅需传递适配器即可,无需转移优化器与调度器状态,关键约束为单步显存和单次会话时长,而非总计算量。更重要的是,评估显示警示性结果:在盲测中,微调模型相较基线在合成数据分布相似度(BERTScore F1 +0.063)上更高,但在实际建议质量上更低——盲评中46%提示更偏好基线模型,而微调模型仅18%;事实核查发现微调模型在政策敏感话题中存在4处确定性错误,基线为零。进一步审计训练数据发现,这些错误早存在于Gemini生成的答案中,随机抽样验证显示28%-40%的回答含可核实错误(单评委,n=40)。因此,错误源于合成数据管道,而非适配器接力方法。我们公开数据集、适配器、跨显卡笔记本及完整评估工具链,确保所有结果可在单块16GB GPU上复现。
原文摘要 · Abstract (English)
Fine-tuning a 7B language model for specialized advising is attractive in resource-constrained settings, but multi-epoch runs routinely exceed the wall-clock limits of the free-tier GPUs (Kaggle, Colab) such users rely on. We report two things. First, a practical recipe: a three-epoch QLoRA fine-tune of Mistral-7B-Instruct-v0.3 (4-bit NF4, LoRA rank 16, via Unsloth) completed across two free-tier 16 GB GPUs (Tesla P100 then T4) by checkpointing only the small LoRA adapter (41.9M parameters) and resuming on the second machine. Adapter-only handoff is sufficient -- optimizer and scheduler state need not be transferred -- so the binding constraint is per-step VRAM and per-session wall-clock, not aggregate compute. Second, and more importantly, an honest evaluation that returns a cautionary result. On a blind held-out comparison against the un-fine-tuned base model, the fine-tuned model scored higher on similarity to the synthetic training distribution (BERTScore F1 +0.063, a fidelity not quality signal) but lower on advising quality: a blind LLM-as-judge preferred the base model on 46% of prompts versus 18%, and a source-verified factuality audit found four confident errors from the fine-tuned model on policy-sensitive topics against zero for the base. Auditing the training data with the same method, we find this is not a fine-tuning artifact: each audited error is already present in the Gemini-generated training answers, and a random-sample audit finds verifiable errors in a sizable fraction of responses (28-40%; single-judge, n=40). The data is therefore sufficient to account for the errors, which we attribute to the synthetic-data pipeline rather than the adapter-handoff method. We release the dataset, adapter, cross-GPU notebooks, and full evaluation harness so every result reproduces on a single 16 GB GPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。