arXiv:2605.20309cs.CVcs.AI2026-05

用关键词索引的微型记忆表,让生成模型精准控制何时调用特定概念。

Tiny-Engram: Trigger-Indexed Concept Tables for Generative Vision

论文配图:Tiny-Engram: Trigger-Indexed Concept Tables for Generative Vision
图 1 · 摘自论文原文
  • 通过触发词匹配的内存表,仅在对应文本区域激活概念
  • 在图像生成中实现精准身份绑定与组合控制,视频生成效果受限
  • 适合需要精确个性化控制的图像生成场景,视频应用待优化

当前生成视觉模型的个性化方法通常通过连续适配器或权重更新编码新概念,但难以控制概念的调用时机。本文提出Tiny-Engram,一种紧凑的触发词索引概念表,在冻结的图像和视频生成器中为视觉记忆赋予显式词汇地址和激活边界。每个概念以一组由注册n-gram匹配索引的小型记忆条目表示,仅在匹配的触发词区域内调节文本编码器隐藏状态。在此范围外,条件路径与冻结基模型一致。在单编码器潜空间扩散模型和多编码器扩散-变压器架构中,该方法成功将罕见触发短语与目标身份绑定,同时保留周围提示的组合控制能力。进一步在文本条件视频生成中评估发现,触发路径可可靠改变生成主体,但跨未见视频提示的身份持续性仍有限。结果表明,小型显式索引概念表是模块化视觉个性化的可行路径,证据最强于图像生成;视频生成中的差距提示:时间上稳定的身份可能需更强的文本侧记忆与动态视觉状态耦合,推动未来工作探索超越文本条件接口的记忆注入。

原文摘要 · Abstract (English)

Current personalization methods for generative vision models typically encode new concepts through continuous adapters or weight updates, yet provide limited control over whether and when a concept should be retrieved. In this work, we introduce Tiny-Engram, a compact trigger-indexed concept table that gives visual memories an explicit lexical address and activation boundary inside frozen image and video generators. Tiny-Engram parameterizes each concept as a small set of memory entries indexed by registered n-gram matches, which modulate text-encoder hidden states only within the matched trigger region. Outside this lexical support, the conditioning pathway is identical to that of the frozen base model. Across both single-encoder latent diffusion and multi-encoder diffusion-transformer backbones, this formulation binds a rare trigger phrase to a target identity while preserving compositional control from the surrounding prompt. We further evaluate the same table-based memory in a text-conditioned video generation setting, where the trigger path reliably alters the generated subject but fine-grained identity persistence across held-out video prompts remains limited. Taken together, these results suggest that small, explicitly addressed concept tables are a practical route to modular visual personalization, with strongest evidence in image generation. For video diffusion, the remaining gap points to a broader requirement: temporally stable identity likely depends on tighter coupling between text-side memory and the evolving visual state, motivating future work on memory injection beyond the text-conditioning interface.

视觉生成个性化记忆机制扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。