将大模型部署到边缘设备,自动生成并验证视频处理命令。
ELLMPEG: An Edge-based Agentic LLM Video Processing Tool
- 用工具感知的检索增强生成+自我反思,本地生成可执行的FFmpeg/VVenC命令
- 在480个测试查询中平均准确率达78%,零云端API费用
- 适合需要隐私保护和低成本视频处理的开发者
大型语言模型(LLMs)正推动生成式AI在多媒体领域的应用,但云部署面临高算力能耗、隐私风险及持续API成本三大挑战。本文提出基于边缘计算的智能体框架ELLMPEG,通过工具感知的检索增强生成(RAG)与迭代自我反思机制,在本地自动生成并验证可执行的FFmpeg与VVenC命令,无需依赖外部云服务。为评估性能,我们构建了包含480个多样化查询的专用提示数据集,覆盖不同类别的FFmpeg与通用视频编码器(VVC)指令。实验对比四种开源LLM在命令有效性、每秒生成词数、推理时延与能效方面的表现,并实际执行生成命令以检验运行正确性与实用性。结果表明,经ELLMPEG增强的Qwen2.5在两个数据集上平均命令生成准确率达78%,显著优于其他模型,且实现零重复API成本。
原文摘要 · Abstract (English)
Large language models (LLMs), the foundation of generative AI systems like ChatGPT, are transforming many fields and applications, including multimedia, enabling more advanced content generation, analysis, and interaction. However, cloud-based LLM deployments face three key limitations: high computational and energy demands, privacy and reliability risks from remote processing, and recurring API costs. Recent advances in agentic AI, especially in structured reasoning and tool use, offer a better way to exploit open and locally deployed tools and LLMs. This paper presents ELLMPEG, an edge-enabled agentic LLM framework for the automated generation of video-processing commands. ELLMPEG integrates tool-aware Retrieval-Augmented Generation (RAG) with iterative self-reflection to produce and locally verify executable FFmpeg and VVenC commands directly at the edge, eliminating reliance on external cloud APIs. To evaluate ELLMPEG, we collect a dedicated prompt dataset comprising 480 diverse queries covering different categories of FFmpeg and the Versatile Video Codec (VVC) encoder (VVenC) commands. We validate command generation accuracy and evaluate four open-source LLMs based on command validity, tokens generated per second, inference time, and energy efficiency. We also execute the generated commands to assess their runtime correctness and practical applicability. Experimental results show that Qwen2.5, when augmented with the ELLMPEG framework, achieves an average command-generation accuracy of 78 % with zero recurring API cost, outperforming all other open-source models across both the FFmpeg and VVenC datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。