本地运行的LLM文档工具，保护隐私且开箱即用
OnPrem.LLM: A Privacy-Conscious Document Intelligence Toolkit
- 本地部署支持多种量化模型，无缝切换后端
- 提供文档处理、RAG、信息提取等预构建流程
- 无代码界面适合非技术人员，兼顾数据安全
我们提出OnPrem.LLM，一个基于Python的工具包，可在离线或受限环境中对敏感非公开数据应用大语言模型。系统专为隐私保护设计，提供文档处理、存储、检索增强生成（RAG）、信息抽取、摘要、分类及提示/输出处理的预构建流水线，配置极简。支持llama.cpp、Ollama、vLLM和Hugging Face Transformers等多种模型后端，具备量化模型支持、GPU加速和无缝后端切换能力。虽以完全本地执行为目标，也允许在合规前提下接入多种云LLM服务商，实现性能与数据控制兼顾的混合部署。配套无代码网页界面，提升非技术用户使用便利性。
原文摘要 · Abstract (English)
We present OnPrem$.$LLM, a Python-based toolkit for applying large language models (LLMs) to sensitive, non-public data in offline or restricted environments. The system is designed for privacy-preserving use cases and provides prebuilt pipelines for document processing and storage, retrieval-augmented generation (RAG), information extraction, summarization, classification, and prompt/output processing with minimal configuration. OnPrem$.$LLM supports multiple LLM backends -- including llama$.$cpp, Ollama, vLLM, and Hugging Face Transformers -- with quantized model support, GPU acceleration, and seamless backend switching. Although designed for fully local execution, OnPrem$.$LLM also supports integration with a wide range of cloud LLM providers when permitted, enabling hybrid deployments that balance performance with data control. A no-code web interface extends accessibility to non-technical users.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。