PARA-Drive: Parallelized Architecture for Real-time Autonomous Driving

Abstract

Recent works have proposed end-to-end autonomous vehicle (AV) architectures comprised of differentiable modules, achieving state-of-the-art driving performance. While they provide advantages over the traditional perception-prediction-planning pipeline (e.g., removing information bottlenecks between components and alleviating integration challenges), they do so using a diverse combination of tasks, modules, and their interconnectivity. As of yet, however, there has been no systematic analysis of the necessity of these modules or the impact of their connectivity, placement, and internal representations on overall driving performance. Addressing this gap, our work conducts a comprehensive exploration of the design space of end-to-end modular AV stacks. Our findings culminate in the development of PARA-Drive: a fully parallel end-to-end AV architecture. PARA-Drive not only achieves state-of-the-art performance in perception, prediction, and planning, but also significantly enhances runtime speed by nearly 3x, without compromising on interpretability or safety.

Approach

PARA-Drive Overview — **PARA-Drive architecture.** Perception, prediction, and planning modules are co-trained in parallel. No dependency is introduced between modules and information passing between modules is conducted implicitly via the tokenized BEV query features. Runtime speed can be boosted by deactivating modules in text.

Video Results

Citation


@inproceedings{Weng2024paradrive,
    title={PARA-Drive: Parallelized Architecture for Real-time Autonomous Driving},
    author={Weng, Xinshuo and Ivanovic, Boris and Wang, Yan and Wang, Yue and Pavone, Marco},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2024}
}