arXiv:2608.27367cs.CVcs.AI2026-08

让视觉变压器随任务复杂度自动增宽加深,省资源还更准。

Successive Capacity Growth: Task-Complexity-Driven Width and Depth Expansion for Vision Transformer Encoders in JEPA World Models

论文配图:Successive Capacity Growth: Task-Complexity-Driven Width and Depth Expansion for Vision Transformer Encoders in JEPA World Models
图 1 · 摘自论文原文
  • 从极小模型开始,按任务需求逐步增加层数或注意力头数。
  • 在多物体动态任务中比固定小模型提升20.3%预测精度,参数效率高56倍。
  • 无需预设最大容量,适合需要动态适应的机器人和世界建模场景。

联合嵌入预测架构(JEPAs)用于世界建模时,通常使用固定大小的视觉变换器编码器,对简单任务过度配置,对复杂任务又配置不足,且注意力头间存在显著冗余。本文提出逐级容量增长(SCG)方法,从极小编码器(1个注意力头、2层、28.3万参数)出发,根据任务复杂度驱动,通过添加注意力头(增强低层次语义能力)或增加变换块(提升高层次语义抽象),实现宽度或深度的增量扩展。该过程由无任务依赖的测试-验证机制支持,利用函数保持扩展安全地试错,并在性能未提升时回滚。所提出的粗略各向同性高斯正则化(SIGReg)确保所有学习语义维度保持统计独立并契合预测目标,防止扩展过程中出现坍缩。在60维多物体动力学任务中,SCG自然触发深度扩展,相比固定小模型预测损失降低20.3%,参数效率比扩展至固定大模型高出56倍;在2D导航任务中,单次宽度扩展即超越固定大模型23%。在三个复杂度递增的测试环境中,自适应编码器均匹配或超过固定小模型,且零误扩张,函数完全保留(比例=1.0,绝对差=0.0)。结论是:JEPA世界模型编码器无需预先分配最大容量,可随任务需求逐步生长,在保证表征质量的同时显著提升计算与数据效率。

原文摘要 · Abstract (English)

Joint-Embedding Predictive Architectures (JEPAs) for world modeling typically employ fixed-size Vision Transformer encoders that are over-provisioned for simple tasks and under-provisioned for complex ones, with significant redundancy across attention heads. We propose Successive Capacity Growth (SCG), a method that starts from a minimal encoder (1 head, 2 layers, 283K parameters) and grows incrementally in width (adding attention heads for low-level semantic capacity) or depth (adding transformer blocks for higher-order semantic abstraction), driven by a task-agnostic test-and-verify mechanism that exploits function-preserving expansion to safely trial architectural changes and roll back if they do not improve prediction loss. The Sketched Isotropic Gaussian Regularizer (SIGReg) ensures that all learned semantic dimensions remain statistically independent and aligned with the predictive objective, preventing collapse even as the architecture grows. On a 60-dimensional multi-object dynamics task, SCG naturally triggers depth expansion, improving prediction loss by 20.3% over the fixed small baseline with 56 times greater parameter efficiency than scaling to the fixed large model; on a 2D navigation task, a single width expansion yields even an 23% improvement over the fixed large model. Across all three tested environments of increasing complexity, the adaptive encoder matches or exceeds the fixed small baseline, with zero false-positive expansions and bit-exact function preservation (ratio = 1.0, absolute difference = 0.0). The take-away is that JEPA world model encoders need not be pre-allocated at maximum capacity - they can grow successively as the task demands, achieving significant compute and data efficiency while maintaining representation quality.

视觉变换器自适应结构世界模型容量增长

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。