arXiv:2604.08355cs.AI2026-04

用语言模型实现零样本任务迁移,让智能体跨任务复用策略。

ASPECT:Analogical Semantic Policy Execution via Language Conditioned Transfer

论文配图:ASPECT:Analogical Semantic Policy Execution via Language Conditioned Transfer
图 1 · 摘自论文原文
  • 用文本条件变分自编码器替代离散变量,通过语言描述动态对齐任务
  • 在复杂新任务上实现零样本迁移,无需预设类别体系
  • 适合需要灵活泛化能力的强化学习场景,尤其擅长处理组合型任务

强化学习智能体在面对结构相似的新任务时仍难以泛化。尽管已有方法尝试通过零样本迁移缓解此问题,但常受限于预定义的离散类别体系,难以适应新颖或组合式任务变化。本文提出一种更通用的方法:将离散潜在变量替换为自然语言条件,利用文本条件变分自编码器(VAE)建模。核心创新在于测试时引入大语言模型(LLM)作为动态语义算子,不依赖固定规则,而是让智能体通过查询LLM,将当前观测的语义描述映射到源任务语境中。该对齐后的描述作为条件输入VAE,生成与原始训练状态兼容的想象状态,从而直接复用原策略。借助LLM的灵活推理能力,本方法实现了对广泛复杂且真正新颖的类比任务的零样本迁移,突破了固定类别映射的局限。代码与视频见链接。

原文摘要 · Abstract (English)

Reinforcement Learning (RL) agents often struggle to generalize knowledge to new tasks, even those structurally similar to ones they have mastered. Although recent approaches have attempted to mitigate this issue via zero-shot transfer, they are often constrained by predefined, discrete class systems, limiting their adaptability to novel or compositional task variations. We propose a significantly more generalized approach, replacing discrete latent variables with natural language conditioning via a text-conditioned Variational Autoencoder (VAE). Our core innovation utilizes a Large Language Model (LLM) as a dynamic \textit{semantic operator} at test time. Rather than relying on rigid rules, our agent queries the LLM to semantically remap the description of the current observation to align with the source task. This source-aligned caption conditions the VAE to generate an imagined state compatible with the agent's original training, enabling direct policy reuse. By harnessing the flexible reasoning capabilities of LLMs, our approach achieves zero-shot transfer across a broad spectrum of complex and truly novel analogous tasks, moving beyond the limitations of fixed category mappings. Code and videos are available \href{https://anonymous.4open.science/r/ASPECT-85C3/}{here}.

强化学习零样本迁移大语言模型策略复用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。