将量化大模型部署到机器人端，实现低资源下的自然语言交互与决策。
Integrating Quantized LLMs into Robotics Systems as Edge AI to Leverage their Natural Language Processing Capabilities
- 基于llama.cpp构建ROS 2工具链，支持量化LLM在机器人边缘运行
- 在资源受限环境下实现高效推理，提升机器人自然语言理解能力
- 适合需要自然语言交互的自主机器人开发，尤其关注可解释性与规划
大型语言模型（LLMs）在过去一年中取得显著进展，广泛应用于自然语言任务。将其集成到机器人系统中，可提升人机交互、导航、规划与决策能力。本文提出llama_ros，一个基于ROS 2的工具，用于在机器人系统中集成量化大型语言模型。依托llama.cpp这一高度优化的运行时引擎，llama_ros实现了在资源受限环境下的高效量化LLM边缘计算，解决了计算效率与内存限制的挑战。通过部署量化LLM，llama_ros使机器人能够利用自然语言理解与生成能力，增强决策与交互性能，并可结合提示工程、知识图谱或本体等工具进一步提升自主机器人能力。此外，本文还探讨了llama_ros在机器人规划与可解释性方面的应用案例。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have experienced great advancements in the last year resulting in an increase of these models in several fields to face natural language tasks. The integration of these models in robotics can also help to improve several aspects such as human-robot interaction, navigation, planning and decision-making. Therefore, this paper introduces llama\_ros, a tool designed to integrate quantized Large Language Models (LLMs) into robotic systems using ROS 2. Leveraging llama.cpp, a highly optimized runtime engine, llama\_ros enables the efficient execution of quantized LLMs as edge artificial intelligence (AI) in robotics systems with resource-constrained environments, addressing the challenges of computational efficiency and memory limitations. By deploying quantized LLMs, llama\_ros empowers robots to leverage the natural language understanding and generation for enhanced decision-making and interaction which can be paired with prompt engineering, knowledge graphs, ontologies or other tools to improve the capabilities of autonomous robots. Additionally, this paper provides insights into some use cases of using llama\_ros for planning and explainability in robotics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。