让医疗AI像医生一样一步步推理,还能自动调节思考深度。
RVLM: Recursive Vision-Language Models with Adaptive Depth
- 用循环生成代码+调用视觉工具的方式逐步推理
- 在脑部和胸部影像上准确识别病灶并发现跨模态矛盾
- 能根据任务难易自动调整思考次数,既省算力又不马虎
医疗AI面临两大瓶颈:传统视觉语言模型仅单次推断,结果不可审计;而固定迭代次数的推理系统对简单任务浪费算力,对复杂任务又不够深入。本文提出统一框架RVLM,将单次推断改为生成-执行循环:每步生成可执行的Python代码,调用视觉子代理、操作图像并积累证据,确保每个诊断结论都有可追溯的代码支撑,满足临床治理要求。同时引入轻量级控制器RRouter,根据任务复杂度特征预测最优迭代预算,并在推理停滞时提前终止。在BraTS 2023 Meningioma(脑MRI)与MIMIC-CXR(胸片)数据集上,使用Gemini 2.5 Flash未微调即实现高一致性检测(如占位性病变及强化表现),并成功识别FLAIR信号特征与分割边界间的跨模态差异。在MIMIC-CXR上可生成结构化报告并正确识别体位相关伪影。
原文摘要 · Abstract (English)
Medical AI systems face two fundamental limitations. First, conventional vision-language models (VLMs) perform single-pass inference, yielding black-box predictions that cannot be audited or explained in clinical terms. Second, iterative reasoning systems that expose intermediate steps rely on fixed iteration budgets wasting compute on simple cases while providing insufficient depth for complex ones. We address both limitations with a unified framework. RVLM replaces single-pass inference with an iterative generate-execute loop: at each step, the model writes Python code, invokes vision sub-agents, manipulates images, and accumulates evidence. Every diagnostic claim is grounded in executable code, satisfying auditability requirements of clinical AI governance frameworks. RRouter makes iteration depth adaptive: a lightweight controller predicts the optimal budget from task-complexity features, then monitors progress and terminates early when reasoning stalls. We evaluate on BraTS 2023 Meningioma (brain MRI) and MIMIC-CXR (chest X-ray) using Gemini 2.5 Flash without fine-tuning. Across repeated runs, RVLM shows high consistency on salient findings (e.g., mass presence and enhancement) and can detect cross-modal discrepancies between Fluid-Attenuated Inversion Recovery (FLAIR) signal characteristics and segmentation boundaries. On MIMIC-CXR, it generates structured reports and correctly recognises view-specific artefacts. Code: https://github.com/nican2018/rvlm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。