arXiv:2511.01594cs.ROcs.CV2025-11被引 1

用多智能体系统让机器人在家中更安全、个性化地协助残障人士。

MARS: Multi-Agent Robotic System with Multimodal Large Language Models for Assistive Intelligence

  • 四智能体协作:感知、风险评估、规划、评估,分层决策。
  • 在多个数据集上表现优于现有模型,尤其在风险感知和协同执行上。
  • 适合需要智能助行、居家照护的残障人士及相关研究者。

多模态大语言模型(MLLMs)在跨模态理解与推理方面展现出强大能力,为智能辅助系统带来新机遇,但现有系统仍面临风险感知规划不足、用户个性化差、语言指令难以转化为可执行动作等问题。本文提出MARS——基于MLLM的多智能体机器人系统,专为支持残障人士的智能家居机器人设计。系统包含四个智能体:视觉感知智能体从环境图像中提取语义与空间特征,风险评估智能体识别并优先排序潜在危险,规划智能体生成可执行的动作序列,评估智能体实现迭代优化。通过融合多模态感知与分层多智能体决策,该框架可在动态室内环境中实现自适应、风险感知与个性化的辅助。在多个数据集上的实验表明,相比最先进的多模态模型,该系统在风险感知规划与多智能体协同执行方面表现更优。本方法还展示了协作式AI在实际辅助场景中的潜力,并为部署MLLM驱动的多智能体系统提供了可推广的方法论。

原文摘要 · Abstract (English)

Multimodal large language models (MLLMs) have shown remarkable capabilities in cross-modal understanding and reasoning, offering new opportunities for intelligent assistive systems, yet existing systems still struggle with risk-aware planning, user personalization, and grounding language plans into executable skills in cluttered homes. We introduce MARS - a Multi-Agent Robotic System powered by MLLMs for assistive intelligence and designed for smart home robots supporting people with disabilities. The system integrates four agents: a visual perception agent for extracting semantic and spatial features from environment images, a risk assessment agent for identifying and prioritizing hazards, a planning agent for generating executable action sequences, and an evaluation agent for iterative optimization. By combining multimodal perception with hierarchical multi-agent decision-making, the framework enables adaptive, risk-aware, and personalized assistance in dynamic indoor environments. Experiments on multiple datasets demonstrate the superior overall performance of the proposed system in risk-aware planning and coordinated multi-agent execution compared with state-of-the-art multimodal models. The proposed approach also highlights the potential of collaborative AI for practical assistive scenarios and provides a generalizable methodology for deploying MLLM-enabled multi-agent systems in real-world environments.

多智能体辅助机器人风险感知大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。