用AI自动分析街景图,让研究者高效评估社区环境特征。
StreetLens: Enabling Human-Centered AI Agents for Neighborhood Assessment from Street View Imagery
- 通过自然语言提示将社会科学研究经验融入视觉语言模型
- 可同时识别客观数据(如车辆数)与主观感知(如混乱感)
- 适合城市规划、公共卫生等领域的研究人员快速开展实地调研
传统社区研究依赖访谈、问卷和人工标注,虽深入但耗时且需专家参与。近年来,视觉语言模型(VLMs)开始辅助自动化分析,但多为临时方案,缺乏跨研究设计与地理场景的适应性。本文提出StreetLens,一种用户可配置的人机协作工作流,将社会科学研究知识嵌入VLM,实现可扩展的社区环境评估。该系统模拟训练有素的人类编码员,基于成熟访谈协议提取相关街景图像(SVI),并生成从客观特征(如车辆数量)到主观感知(如混乱感)的广泛语义标注。研究者可通过领域引导的提示定义VLM角色,使领域知识成为分析核心。系统还可融合前期调查数据,提升结果稳健性并拓展评估维度。StreetLens代表了向灵活、智能代理型AI系统的转变,助力研究者加速和扩大社区研究规模。项目已开源:https://knowledge-computing.github.io/projects/streetlens。
原文摘要 · Abstract (English)
Traditionally, neighborhood studies have used interviews, surveys, and manual image annotation guided by detailed protocols to identify environmental characteristics, including physical disorder, decay, street safety, and sociocultural symbols, and to examine their impact on developmental and health outcomes. Although these methods yield rich insights, they are time-consuming and require intensive expert intervention. Recent technological advances, including vision language models (VLMs), have begun to automate parts of this process; however, existing efforts are often ad hoc and lack adaptability across research designs and geographic contexts. In this paper, we present StreetLens, a user-configurable human-centered workflow that integrates relevant social science expertise into a VLM for scalable neighborhood environmental assessments. StreetLens mimics the process of trained human coders by focusing the analysis on questions derived from established interview protocols, retrieving relevant street view imagery (SVI), and generating a wide spectrum of semantic annotations from objective features (e.g., the number of cars) to subjective perceptions (e.g., the sense of disorder in an image). By enabling researchers to define the VLM's role through domain-informed prompting, StreetLens places domain knowledge at the core of the analysis process. It also supports the integration of prior survey data to enhance robustness and expand the range of characteristics assessed in diverse settings. StreetLens represents a shift toward flexible and agentic AI systems that work closely with researchers to accelerate and scale neighborhood studies. StreetLens is publicly available at https://knowledge-computing.github.io/projects/streetlens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。