让摄像头3D目标检测模型能高效部署，精度几乎不降。
FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detection
- 用单点采样和锚框嵌入替代多点采样，解决特征尺度差异问题。
- 用双查表法逼近非线性运算，仅需少量查表项且无需特殊硬件。
- 先稳定softmax再量化，缓解大输入导致的注意力失真。
基于摄像头的多视角3D目标检测对自动驾驶至关重要。PETR及其变体在基准测试中表现优异，但因计算成本高、内存占用大而难以部署。量化通过降低权重和激活值的位宽来压缩神经网络，但直接应用于PETRs会导致严重精度下降。主要挑战包括：（1）多模态特征间显著的幅度差异——特别是图像特征与相机射线位置嵌入（PE）之间；（2）非线性算子量化效率低且存在近似误差，常依赖硬件不友好的计算。本文提出FQ-PETR，一种面向PETRs的全量化框架，包含三项创新：（1）量化友好型LiDAR射线位置嵌入（QFPE）：以LiDAR先验引导的单点采样和锚框嵌入取代多点采样，消除反sigmoid等非线性操作，使PE尺度与图像特征对齐，保持精度；（2）双查表（DULUT）：通过两级线性查表逼近复杂非线性函数，实现高保真度且仅需极少条目，无需专用硬件；（3）量化后数值稳定性处理（QANS）：在softmax数值稳定后进行量化，减轻大输入引发的注意力畸变。在PETR、StreamPETR、PETRv2、MV2d等模型上，FQ-PETR实现W8A8量化时仅损失1%精度，延迟最高降低75%，显著优于现有后训练量化（PTQ）和量化感知训练（QAT）基线。
原文摘要 · Abstract (English)
Camera-based multi-view 3D detection is crucial for autonomous driving. PETR and its variants (PETRs) excel in benchmarks but face deployment challenges due to high computational cost and memory footprint. Quantization is an effective technique for compressing deep neural networks by reducing the bit width of weights and activations. However, directly applying existing quantization methods to PETRs leads to severe accuracy degradation. This issue primarily arises from two key challenges: (1) significant magnitude disparity between multi-modal features-specifically, image features and camera-ray positional embeddings (PE), and (2) the inefficiency and approximation error of quantizing non-linear operators, which commonly rely on hardware-unfriendly computations. In this paper, we propose FQ-PETR, a fully quantized framework for PETRs, featuring three key innovations: (1) Quantization-Friendly LiDAR-ray Position Embedding (QFPE): Replacing multi-point sampling with LiDAR-prior-guided single-point sampling and anchor-based embedding eliminates problematic non-linearities (e.g., inverse-sigmoid) and aligns PE scale with image features, preserving accuracy. (2) Dual-Lookup Table (DULUT): This algorithm approximates complex non-linear functions using two cascaded linear LUTs, achieving high fidelity with minimal entries and no specialized hardware. (3) Quantization After Numerical Stabilization (QANS): Performing quantization after softmax numerical stabilization mitigates attention distortion from large inputs. On PETRs (e.g. PETR, StreamPETR, PETRv2, MV2d), FQ-PETR under W8A8 achieves near-floating-point accuracy (1% degradation) while reducing latency by up to 75%, significantly outperforming existing PTQ and QAT baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。