在8GB显存下，用RTX 4060实测高效微调大模型的方案与瓶颈。
Profiling LoRA/QLoRA Fine-Tuning Efficiency on Consumer GPUs: An RTX 4060 Case Study
- 对比不同参数配置下的微调效率，重点测试了分页优化器和精度设置。
- 分页优化器使吞吐量提升25%（最高628 tokens/s），bf16比fp16更慢。
- 8GB显存下仍可支持最长2048词元的序列，适合资源有限的研究者参考。
使用参数高效技术如LoRA和QLoRA微调大语言模型（LLMs）使得在普通硬件上适配基础模型成为可能。然而，在消费级GPU上，特别是在严格的8 GB VRAM限制下，此类训练的效率仍缺乏系统研究。本研究以单块NVIDIA RTX 4060为平台，对Qwen2.5-1.5B-Instruct模型进行受控的微调效率分析，系统调整批大小、序列长度、优化器选择（AdamW vs. PagedAdamW）及精度（fp16 vs. bf16）。报告了吞吐量（tokens/s）、每1万词元耗时及显存占用，并基于显卡功耗上限估算能耗。结果表明，分页优化器可将吞吐量提升至多25%（628 tok/s vs. 500 tok/s基线），而bf16相较fp16效率下降。尽管受限于8 GB显存，采用参数高效策略仍可实现长达2048词元的序列处理。据我们所知，这是首个针对消费级GPU上的大模型微调效率的系统性案例研究，提供了可复现的基准数据与实用指南，供资源受限的研究人员和实践者参考。
原文摘要 · Abstract (English)
Fine-tuning large language models (LLMs) with parameter-efficient techniques such as LoRA and QLoRA has enabled adaptation of foundation models on modest hardware. Yet the efficiency of such training on consumer-grade GPUs, especially under strict 8 GB VRAM limits, remains underexplored. We present a controlled profiling study of LoRA/QLoRA fine-tuning using the Qwen2.5-1.5B-Instruct model on a single NVIDIA RTX 4060. Across three representative configurations, we systematically vary batch size, sequence length, optimizer choice (AdamW vs. PagedAdamW), and precision (fp16 vs. bf16). We report throughput (tokens/s), time per 10k tokens, and VRAM footprint, alongside energy estimates derived from GPU board power limits. Our results show that paged optimizers improve throughput by up to 25% (628 tok/s vs. 500 tok/s baseline), while bf16 degrades efficiency relative to fp16. Despite 8 GB constraints, sequence lengths up to 2048 tokens were feasible using parameter-efficient strategies. To our knowledge, this is the first systematic case study of LLM fine-tuning efficiency on consumer GPUs, providing reproducible benchmarks and practical guidelines for resource-constrained researchers and practitioners.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。