让3D目标检测模型在低精度下仍保持高精度，适合自动驾驶部署。
FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detection
- 用单点采样和锚点嵌入替代多点采样，解决特征尺度差异问题。
- 通过双查表法逼近非线性函数，仅用少量查表项实现高精度。
- 先稳定softmax再量化，避免注意力机制因大输入失真。
基于摄像头的多视角3D目标检测对自动驾驶至关重要。PETR及其变体在基准测试中表现优异，但因计算成本高、内存占用大而难以部署。量化可通过降低权重和激活位宽压缩神经网络，但直接应用于PETR会导致严重精度下降。主要原因有两个：一是多模态特征间存在显著量级差异，特别是图像特征与相机射线位置嵌入（PE）之间；二是非线性算子量化效率低且存在近似误差，常依赖硬件不友好的计算。本文提出FQ-PETR，一种面向PETR的全量化框架，包含三项创新：(1) 量化友好型LiDAR射线位置嵌入（QFPE）：以LiDAR先验引导的单点采样和锚点嵌入替代多点采样，消除反sigmoid等非线性问题，使PE量级与图像特征对齐，保留精度；(2) 双查表法（DULUT）：通过两级级联线性查表逼近复杂非线性函数，仅需少量查表项即可实现高保真度，无需专用硬件；(3) 数值稳定后量化（QANS）：在softmax数值稳定后再进行量化，缓解大输入导致的注意力失真。在PETR系列模型（如PETR、StreamPETR、PETRv2、MV2d）上，FQ-PETR在W8A8条件下达到接近浮点精度（仅1%退化），同时延迟降低最高达75%，显著优于现有定点量化（PTQ）和量化感知训练（QAT）基线。
原文摘要 · Abstract (English)
Camera-based multi-view 3D detection is crucial for autonomous driving. PETR and its variants (PETRs) excel in benchmarks but face deployment challenges due to high computational cost and memory footprint. Quantization is an effective technique for compressing deep neural networks by reducing the bit width of weights and activations. However, directly applying existing quantization methods to PETRs leads to severe accuracy degradation. This issue primarily arises from two key challenges: (1) significant magnitude disparity between multi-modal features-specifically, image features and camera-ray positional embeddings (PE), and (2) the inefficiency and approximation error of quantizing non-linear operators, which commonly rely on hardware-unfriendly computations. In this paper, we propose FQ-PETR, a fully quantized framework for PETRs, featuring three key innovations: (1) Quantization-Friendly LiDAR-ray Position Embedding (QFPE): Replacing multi-point sampling with LiDAR-prior-guided single-point sampling and anchor-based embedding eliminates problematic non-linearities (e.g., inverse-sigmoid) and aligns PE scale with image features, preserving accuracy. (2) Dual-Lookup Table (DULUT): This algorithm approximates complex non-linear functions using two cascaded linear LUTs, achieving high fidelity with minimal entries and no specialized hardware. (3) Quantization After Numerical Stabilization (QANS): Performing quantization after softmax numerical stabilization mitigates attention distortion from large inputs. On PETRs (e.g. PETR, StreamPETR, PETRv2, MV2d), FQ-PETR under W8A8 achieves near-floating-point accuracy (1% degradation) while reducing latency by up to 75%, significantly outperforming existing PTQ and QAT baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。