Abstract:

Bird’s-eye-view (BEV) perception is a core technology for autonomous driving systems. However, existing solutions face the dilemma of high costs associated with multi-modal methods and limited performance of vision-only approaches. To address this issue, this paper proposes a framework named “a lightweight pure visual BEV perception method based on dual distillation of spatial-temporal knowledge”. This framework innovatively designs a lightweight vision-only student model based on ResNet, which leverages a dual distillation mechanism to learn from a powerful teacher model that integrates temporal information from both image and light detection and ranging (LiDAR) modalities. Specifically, we distill efficient multi-modal feature extraction and spatial fusion capabilities from the BEVFusion model, and distill advanced temporal information fusion and spatiotemporal attention mechanisms from the BEVFormer model. This dual distillation strategy enables the student model to achieve perception performance close to that of multi-modal models without relying on LiDAR. Experimental results on the nuScenes dataset demonstrate that the proposed model significantly outperforms classical vision-only algorithms, achieves comparable performance to current state-of-the-art vision-only methods on the nuScenes detection leaderboard in terms of both mean average precision (mAP) and the nuScenes detection score (NDS) metrics, and exhibits notable advantages in inference computational efficiency. Although the proposed dual-teacher paradigm incurs higher offline training costs compared to single-model approaches, it yields a streamlined and highly efficient student model suitable for resource-constrained real-time deployment. This provides an effective pathway toward low-cost, high-performance autonomous driving perception systems.

Key words: 3D object detection, bird’s-eye-view (BEV), knowledge distillation, multimodal fusion, lightweight model