综述大模型在跨模态推理中的方法与挑战，助你快速掌握前沿进展。
From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Models
- 构建三层次分类体系，系统梳理大模型跨模态推理方法
- 分析典型模型设计策略与关键技术实现路径
- 适合研究者快速了解领域脉络与未来方向
跨模态推理（CMR）是融合不同感官模态信息并进行推断的复杂过程，被视为迈向更智能、类人人工智能的关键能力。大型语言模型（LLMs）专为大规模处理与生成人类语言而设计。近年来将LLMs应用于CMR任务已成为主流趋势，显著提升了性能。本文系统综述了当前基于LLMs的跨模态推理方法，提出一个详尽的三层分类体系；深入剖析代表性模型的核心设计策略与运行机制；同时指出当前整合中面临的主要挑战，并展望未来研究方向。本综述旨在通过提供全面、深入的视角，加速该新兴领域的发展。相关论文资源可访问GitHub仓库：https://github.com/ZuyiZhou/Awesome-Cross-modal-Reasoning-with-LLMs
原文摘要 · Abstract (English)
Cross-modal reasoning (CMR), the intricate process of synthesizing and drawing inferences across divergent sensory modalities, is increasingly recognized as a crucial capability in the progression toward more sophisticated and anthropomorphic artificial intelligence systems. Large Language Models (LLMs) represent a class of AI algorithms specifically engineered to parse, produce, and engage with human language on an extensive scale. The recent trend of deploying LLMs to tackle CMR tasks has marked a new mainstream of approaches for enhancing their effectiveness. This survey offers a nuanced exposition of current methodologies applied in CMR using LLMs, classifying these into a detailed three-tiered taxonomy. Moreover, the survey delves into the principal design strategies and operational techniques of prototypical models within this domain. Additionally, it articulates the prevailing challenges associated with the integration of LLMs in CMR and identifies prospective research directions. To sum up, this survey endeavors to expedite progress within this burgeoning field by endowing scholars with a holistic and detailed vista, showcasing the vanguard of current research whilst pinpointing potential avenues for advancement. An associated GitHub repository that collects the relevant papers can be found at https://github.com/ZuyiZhou/Awesome-Cross-modal-Reasoning-with-LLMs
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。