用统一框架让大模型像函数一样编程，支持多模态与逻辑结合。
Relational Programming with Foundation Models
- 基于概率关系范式，将大模型视为无状态函数处理输入输出。
- 在9个跨语言、视觉和数据库任务中表现媲美或优于基线。
- 支持12种大模型插件，可无缝组合代码与逻辑程序。
基础模型具备广泛潜力以支持多样化的AI应用。其强大但不完整的特点催生了多种增强机制，如上下文学习、信息检索和代码解释。我们提出Vieira，一种声明式框架，将这些机制统一为通用的大模型编程解决方案。Vieira遵循概率关系范式，将基础模型视为具有关系型输入输出的无状态函数。它通过融合神经符号系统支持逻辑程序，同时通过简化多样子模型的组合来实现复杂多模态应用。我们通过扩展Scallop编译器，引入支持基础模型作为插件的外部接口，实现了对12个基础模型（包括GPT、CLIP和SAM）的插件支持。我们在9个涵盖语言、视觉以及结构化和向量数据库的挑战性任务上评估了Vieira。结果表明，Vieira中的程序简洁高效，能集成现代基础模型，并在准确率上达到或超越竞争基线。
原文摘要 · Abstract (English)
Foundation models have vast potential to enable diverse AI applications. The powerful yet incomplete nature of these models has spurred a wide range of mechanisms to augment them with capabilities such as in-context learning, information retrieval, and code interpreting. We propose Vieira, a declarative framework that unifies these mechanisms in a general solution for programming with foundation models. Vieira follows a probabilistic relational paradigm and treats foundation models as stateless functions with relational inputs and outputs. It supports neuro-symbolic applications by enabling the seamless combination of such models with logic programs, as well as complex, multi-modal applications by streamlining the composition of diverse sub-models. We implement Vieira by extending the Scallop compiler with a foreign interface that supports foundation models as plugins. We implement plugins for 12 foundation models including GPT, CLIP, and SAM. We evaluate Vieira on 9 challenging tasks that span language, vision, and structured and vector databases. Our evaluation shows that programs in Vieira are concise, can incorporate modern foundation models, and have comparable or better accuracy than competitive baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。