Flama统一开发部署机器学习与大模型服务的生产级Python框架
Flama: a Python framework for development and deployment of production-ready APIs, machine learning, and LLM services
- 基于ASGI的异步编程模型，通过类型注解自动解析依赖
- 支持scikit-learn、PyTorch等模型零代码打包为. flm二进制格式
- 兼容vLLM与MLX后端，提供OpenAI/Anthropic等多协议接口
我们提出Flama，一个开源的Python框架，用于开发和部署生产级Web API、机器学习服务及大语言模型（LLM）应用。基于异步服务器网关接口（ASGI），Flama采用类型驱动、异步优先的编程模型，将REST API开发、预测模型服务与生成式AI推理统一在一个架构中。其由七个子系统构成：基于组件的依赖注入系统在启动时通过类型注解解析处理器参数；可插拔的模式层支持Pydantic、Marshmallow和Typesystem，通过单一适配器统一接口；自动CRUD生成器将SQLAlchemy表与模式类转化为基于仓库和工作单元模式的REST端点；便携式二进制格式（.flm）打包scikit-learn、TensorFlow、PyTorch及Hugging Face Transformers模型及其元数据，实现零代码部署；多后端LLM服务器可在vLLM（Linux/CUDA）或MLX（Apple Silicon）上运行，并通过共享编解码器暴露OpenAI、Anthropic、Ollama及原生流式协议；经Maturin编译的Rust加速核心处理路由、JSON编码、压缩与解析；模型上下文协议模块使任意应用可通过JSON-RPC 2.0变为MCP服务器。内置功能包括JWT认证、两种分页策略、线程或进程后台任务、WebSocket端点、服务器发送事件与NDJSON流、从处理器签名自动生成OpenAPI 3.2.0文档，以及用于运行、服务、打包与检查模型的命令行接口。本文描述了架构，通过实例展示编程模型，并对比现有框架、模型服务平台与LLM推理引擎。
原文摘要 · Abstract (English)
We present Flama, an open-source Python framework for developing and deploying production-ready web APIs, machine learning services, and large-language-model (LLM) applications. Built on the Asynchronous Server Gateway Interface (ASGI), Flama offers a type-driven, async-first programming model that unifies REST API development, predictive model serving, and generative AI inference in one architecture. It is organised around seven subsystems: a component-based dependency injection system resolving handler parameters from type annotations at startup; a pluggable schema layer supporting Pydantic, Marshmallow and Typesystem behind a single adapter; an automatic CRUD generator turning a SQLAlchemy table and a schema class into REST endpoints backed by the Repository and Unit of Work patterns; a portable binary format (.flm) packaging models from scikit-learn, TensorFlow, PyTorch and Hugging Face Transformers with their metadata for zero-code deployment; a multi-backend LLM server running vLLM (Linux/CUDA) or MLX (Apple Silicon) and exposing four wire protocols (OpenAI, Anthropic, Ollama, and a native streaming dialect) through a shared codec; a Rust-accelerated core compiled via Maturin for routing, JSON encoding, compression and parsing; and a Model Context Protocol module turning any application into an MCP server over JSON-RPC 2.0. Built-in capabilities include JWT authentication, two pagination strategies, background tasks in threads or processes, WebSocket endpoints, Server-Sent Event and NDJSON streaming, OpenAPI 3.2.0 generation from handler signatures, and a command-line interface for running applications and for serving, packaging and inspecting models. We describe the architecture, present the programming model through worked examples, and compare Flama with existing frameworks, model serving platforms and LLM inference engines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。