让大模型精准调度传感器,实现高效可靠的物理世界感知。
IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling
- 构建时空图结构,将语言指令转为可验证的传感器调度决策
- 在校园级数据集上提升任务成功率37.6%,速度更快、消耗更少
- 适合需要高可靠智能感知的物联网与城市级系统应用
由大规模传感器网络驱动的智能系统正从预设监控转向意图驱动操作,暴露出语义到物理映射的关键鸿沟。尽管大语言模型(LLMs)擅长语义理解,现有以感知为中心的流程多为事后分析,忽略了‘感什么、何时感’这一根本性决策。本文将此主动决策形式化为语义-空间传感器调度(S3),并证明直接用LLM规划不可靠,因其存在表征、推理与优化上的固有缺陷。为此,我们提出空间轨迹图(STG)——一种神经符号范式,遵循‘先验证后执行’原则,将开放式规划转化为可验证的图优化问题。基于此,我们实现了IoT-Brain系统,并构建了TopoSense-Bench,一个涵盖2,510个摄像头、5,250条自然语言查询的校园级基准。评估显示,IoT-Brain相比最强搜索方法任务成功率提升37.6%,运行速度接近两倍快,提示词消耗减少6.6倍。实际部署中,其接近可靠性上限,同时网络带宽降低4.1倍,为大模型与物理世界交互提供了前所未有的高效可靠框架。
原文摘要 · Abstract (English)
Intelligent systems powered by large-scale sensor networks are shifting from predefined monitoring to intent-driven operation, revealing a critical Semantic-to-Physical Mapping Gap. While large language models (LLMs) excel at semantic understanding, existing perception-centric pipelines operate retrospectively, overlooking the fundamental decision of what to sense and when. We formalize this proactive decision as Semantic-Spatial Sensor Scheduling (S3) and demonstrate that direct LLM planning is unreliable due to inherent gaps in representation, reasoning, and optimization. To bridge these gaps, we introduce the Spatial Trajectory Graph (STG), a neuro-symbolic paradigm governed by a verify-before-commit discipline that transforms open-ended planning into a verifiable graph optimization problem. Based on STG, we implement IoT-Brain, a concrete system embodiment, and construct TopoSense-Bench, a campus-scale benchmark with 5,250 natural-language queries across 2,510 cameras. Evaluations show that IoT-Brain boosts task success rate by 37.6% over the strongest search-intensive methods while running nearly 2 times faster and using 6.6 times fewer prompt tokens. In real-world deployment, it approaches the reliability upper bound while reducing 4.1 times network bandwidth, providing a foundational framework for LLMs to interact with the physical world with unprecedented reliability and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。