一个可实时切换任务的图像压缩模型,无需重新训练即可适配不同场景。
UniTAC: Universal Task-Aware Compression via Weighted Distortion Measures

- 用任务重要性向量动态调节压缩过程,实现通用到专用的无缝切换。
- 在0.034 bpp下达到91.4%准确率,仅比专用编码器低1.9%。
- 适用于自动驾驶等需快速响应的任务,适合部署在资源受限设备上。
自动驾驶和机器人等物理人工智能系统需要在有限带宽、延迟和能耗条件下实时传输高维感知信号。由于下游任务随时间变化,为每个任务定制编码器既脆弱又无法在实际中频繁重训。本文提出UniTAC,一种单一学习型图像编码器,支持从通用到任务专用的运行时重构,无需重训。任务以每组件重要性向量形式抽象,例如通过任意下游模型的梯度归因获得,并作为低开销侧信息传输,用于调节编码器与解码器。该模型在广泛随机生成的重要性向量上一次性训练,采用加权重建失真目标;其固定主干网络和人类可读重建结果,通过替换注入向量实现对当前任务的精度引导。我们分析了加权率失真问题,揭示对角加权失真何时具任务一致性及其与任务敏感性的关系。基于此,设计基于视觉变换器(ViT)的编码器,其令牌级条件机制天然支持权重驱动编码。在局部任务中,0.034 bpp下单个UniTAC模型达91.4%准确率,仅比任务专用编码器(93.3%)低1.9%,显著高于通用编码器(76.9%)。
原文摘要 · Abstract (English)
Physical AI systems such as autonomous vehicles and robots rely on timely exchange of high-dimensional sensory signals under tight bandwidth, latency, and energy budgets. Because the task driving downstream decisions evolves over time, a task-specific codec is brittle and retraining one per task is infeasible in the field. We propose UniTAC, a single learned image codec spanning universal (task-agnostic) to task-specialized operation, re-targeted at runtime without retraining. The task is abstracted as a per-component importance vector, derived, e.g., from gradient attribution of any downstream model, and transmitted as low-overhead side information that conditions both encoder and decoder. Trained once over a broad, randomized family of such vectors against weighted-reconstruction distortion, UniTAC keeps a fixed backbone and a single human-viewable reconstruction whose fidelity is steered to the active task by swapping the injected vector. We analyze the underlying weighted rate-distortion problem, characterizing when a diagonal weighted distortion is task-consistent and how weights relate to task sensitivity. Guided by this, we design a Vision Transformer (ViT) codec whose token-level conditioning natively realizes this weight-driven code. On a localized task at 0.034 bpp, a single UniTAC model reaches 91.4% accuracy, only 1.9% below a task-based codec (93.3%) and above universal codecs (76.9%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。