用模板化增强生成程序，让小模型低成本高效学写视觉任务代码
Visual Program Distillation with Template-Based Augmentation
- 将程序拆解为可复用的模板与参数，自动生成训练数据
- 仅需少量问答数据，小模型就能生成高质量专用视觉程序
- 适合资源有限但需快速部署视觉编程系统的团队
将大型语言模型（LLM）用于生成视觉任务如视觉问答（VQA）的可执行代码，面临标注和推理成本高的问题。本文提出一种低成本的视觉程序蒸馏方法，适用于最多10亿参数的模型，且无需人工编写程序标注。通过将程序解耦为高层技能（称为模板）及其对应参数，构建基于模板的合成数据增强。实验表明，仅使用少量问题/答案数据，小型语言模型即可生成高质量的专用视觉程序，同时具备显著更快的推理速度。
原文摘要 · Abstract (English)
Adapting visual programming or prompting large language models (LLMs) to generate executable code for visual tasks like visual question answering (VQA) for specialized tasks or domains remains challenging due to high annotation and inference costs. We propose a low-cost visual program distillation method that can be used for models with at most 1 billion parameters and requires no human-generated program annotations. We achieve this through synthetic data augmentation based on decoupling programs into higher-level skills, called templates, and their corresponding arguments. Experimental results show that, with a relatively small amount of question/answer data, small language models can generate high-quality specialized visual programs with the added benefit of much faster inference
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。