让模型像用记忆一样个性化,不靠改参数,而是实时生成适配新任务的权重。
HY-WU (Part I): An Extensible Functional Neural Memory Framework and An Instantiation in Text-Guided Image Editing
- 用神经生成器动态合成针对不同任务的权重更新,无需测试时优化
- 打破传统单一参数点限制,避免持续学习中旧知识被覆盖
- 适合需要长期适应、用户偏好多变的智能系统应用
基础模型正从离线预测器转向需长期运行的部署系统。实际应用中,目标并非固定:领域会漂移,用户偏好会演变,模型上线后还会出现新任务。这使持续学习和即时个性化从可选功能变为核心架构需求。然而,大多数适应流程仍沿用静态权重范式:训练或适应后,推理始终使用单一参数向量,无视用户意图、领域或实例约束。这将模型视为参数空间中的单一点。在异构且持续演化的环境中,不同目标可能在参数空间中形成分离的可行区域,迫使单一共享更新陷入妥协、干扰或过度专精。结果,持续学习与个性化常通过反复覆盖共享权重实现,危及先前所学行为。我们提出 HY-WU(Weight Unleashing),一种以记忆为核心的适应框架,将适应压力从覆盖单一共享参数点转移。HY-WU 将功能(操作符级)记忆实现为神经模块:一个从实例条件实时生成权重更新的生成器,从而在无需测试时优化的情况下,实现实例特异性操作符。
原文摘要 · Abstract (English)
Foundation models are transitioning from offline predictors to deployed systems expected to operate over long time horizons. In real deployments, objectives are not fixed: domains drift, user preferences evolve, and new tasks appear after the model has shipped. This elevates continual learning and instant personalization from optional features to core architectural requirements. Yet most adaptation pipelines still follow a static weight paradigm: after training (or after any adaptation step), inference executes a single parameter vector regardless of user intent, domain, or instance-specific constraints. This treats the trained or adapted model as a single point in parameter space. In heterogeneous and continually evolving regimes, distinct objectives can induce separated feasible regions over parameters, forcing any single shared update into compromise, interference, or overspecialization. As a result, continual learning and personalization are often implemented as repeated overwriting of shared weights, risking degradation of previously learned behaviors. We propose HY-WU (Weight Unleashing), a memory-first adaptation framework that shifts adaptation pressure away from overwriting a single shared parameter point. HY-WU implements functional (operator-level) memory as a neural module: a generator that synthesizes weight updates on-the-fly from the instance condition, yielding instance-specific operators without test-time optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。