arXiv:2604.09360cs.SEcs.AI2026-04

构建统一中间表示,让不同大模型接口无缝互换。

LLM-Rosetta: A Hub-and-Spoke Intermediate Representation for Cross-Provider LLM API Translation

  • 用9类内容+10类事件的中间表示,抽象共性语义结构。
  • 支持四类主流接口双向转换,微秒级延迟,无信息损失。
  • 模块化设计适合扩展新接口,已用于实际科研系统。

大型语言模型提供商激增,各具私有接口格式,导致应用与厂商深度绑定。在不兼容的接口间切换需$O(N^2)$对适配器,严重阻碍可移植性与多厂商架构。我们发现尽管语法差异显著,主流LLM API共享核心语义:问题在于表面形式组合爆炸,而非深层语义冲突。基于此,提出LLM-Rosetta——一个开源翻译框架,采用中心辐射式中间表示(IR),以9类内容模型与10类流事件模式捕获消息、内容片段、工具调用、推理轨迹及生成控制等共性语义。模块化操作组合转换器架构支持独立添加新标准。框架实现请求与响应的双向转换(含分块流传输与状态管理)。已实现对OpenAI Chat Completions、OpenAI Responses、Anthropic Messages和Google GenAI四类接口的支持,覆盖主要商业服务。实证表明转换具备无损往返保真度、正确流式行为,且延迟低于100微秒,性能媲美LiteLLM单向方案,同时支持双向与厂商中立。通过Open Responses合规测试,并已在阿贡国家实验室生产部署。代码已开源:https://github.com/Oaklight/llm-rosetta。

原文摘要 · Abstract (English)

The rapid proliferation of Large Language Model (LLM) providers--each exposing proprietary API formats--has created a fragmented ecosystem where applications become tightly coupled to individual vendors. Switching or bridging providers requires $O(N^2)$ bilateral adapters, impeding portability and multi-provider architectures. We observe that despite substantial syntactic divergence, the major LLM APIs share a common semantic core: the practical challenge is the combinatorial surface of syntactic variations, not deep semantic incompatibility. Based on this finding, we present LLM-Rosetta, an open-source translation framework built on a hub-and-spoke Intermediate Representation (IR) that captures the shared semantic core--messages, content parts, tool calls, reasoning traces, and generation controls--in a 9-type content model and 10-type stream event schema. A modular Ops-composition converter architecture enables each API standard to be added independently. LLM-Rosetta supports bidirectional conversion (provider-to-IR-to-provider) for both request and response payloads, including chunk-level streaming with stateful context management. We implement converters for four API standards (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google GenAI), covering the vast majority of commercial providers. Empirical evaluation demonstrates lossless round-trip fidelity, correct streaming behavior, and sub-100 microsecond conversion overhead--competitive with LiteLLM's single-pass approach while providing bidirectionality and provider neutrality. LLM-Rosetta passes the Open Responses compliance suite and is deployed in production at Argonne National Laboratory. Code is available at https://github.com/Oaklight/llm-rosetta.

大模型接口中间表示互操作性多厂商

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。