用形式化逻辑压缩大模型，让边缘设备跑得快还靠谱。
TOGGLE: Temporal Logic-Guided Large Language Model Compression for Edge
- 用时序逻辑约束压缩过程，确保语言特性不丢失。
- 压缩后计算量降3.3倍，模型大小减68.8%，仍满足语言要求。
- 无需微调，适合对可靠性要求高的边缘部署场景。
大型语言模型在自然语言任务中表现优异，但计算开销大，难以部署于资源受限的边缘设备。现有压缩技术如量化和剪枝常损害关键语言特性，且缺乏行为保持的形式化保障。本文提出时序逻辑引导的大模型压缩框架TOGGLE，利用信号时序逻辑（STL）形式化定义并强制执行语言属性。TOGGLE采用基于STL鲁棒性的贝叶斯优化，系统探索逐层量化与剪枝配置，在无需重训练或微调的情况下生成满足指定语言约束的压缩模型。在GPT-2、DeepSeek-V2 7B、LLaMA 3 8B和Mistral 7B四类模型上评估，实现最高3.3倍的计算成本（FLOPs）降低，模型大小减少68.8%，同时完全满足所有语言属性。TOGGLE是首个将形式化方法引入大模型压缩的工作，支持高效、可验证的边缘部署。
原文摘要 · Abstract (English)
Large Language Models (LLMs) deliver exceptional performance across natural language tasks but demand substantial computational resources, limiting their deployment on resource-constrained edge devices. Existing compression techniques, such as quantization and pruning, often degrade critical linguistic properties and lack formal guarantees for preserving model behavior. We propose Temporal Logic-Guided Large Language Model Compression (TOGGLE), a novel framework that leverages Signal Temporal Logic (STL) to formally specify and enforce linguistic properties during compression. TOGGLE employs an STL robustness-guided Bayesian optimization to systematically explore layer-wise quantization and pruning configurations, generating compressed models that formally satisfy specified linguistic constraints without retraining or fine-tuning. Evaluating TOGGLE on four LLM architectures (GPT-2, DeepSeek-V2 7B, LLaMA 3 8B, and Mistral 7B), we achieve up to 3.3x reduction in computational costs (FLOPs) and up to a 68.8% reduction in model size while satisfying all linguistic properties. TOGGLE represents the first integration of formal methods into LLM compression, enabling efficient, verifiable deployment of LLMs on edge hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。