6 October 2022 Camera and LiDAR-based point painted voxel region-based convolutional neural network for robust 3D object detection
Han Xie, Wenqi Zheng, Yunfan Chen, Hyunchul Shin
Author Affiliations +
Abstract

Most of the three-dimensional (3D) object detection methods based on LiDAR point cloud data achieve relatively high performance in general cases. However, when the LiDAR points have noise or some corruptions, the detection performance can be severely affected. We propose a 3D object detection method that combines point cloud information with two-dimensional (2D) semantic segmentation information to enhance the feature representation for difficult cases, such as sparse, noisy, and partially absent data. Motivated by the Pointpainting techniques, we designed an early-stage fusion method based on a Voxel region-based convolutional neural network (R-CNN) architecture. The 2D semantic segmentation scores obtained by the Pointpainting techniques are appended to the raw point cloud data. The voxel-based features and 2D semantic information improve the performance in detecting instances when the point cloud is corrupted. In addition, we also designed a multiscale hierarchical region of interest pooling strategy that reduced the computational cost of Voxel R-CNN by at least 43%. Our method shows competitive results with the state-of-the-art methods on the standard KITTI dataset. In addition, three corrupted KITTI datasets, KITTI sparse (KITTI-S), KITTI jittering (KITTI-J), and KITTI dropout (KITTI-D), were used for robustness testing. With the noisy LiDAR points, our proposed point painted Voxel R-CNN achieved superior detection performance over that of the baseline Voxel R-CNN for the moderate case, with a notable improvement of 11.13% in average precision (AP) on the 3D object detection and 14.3% in AP on the bird’s eye view object detection.

© 2022 SPIE and IS&T
Han Xie, Wenqi Zheng, Yunfan Chen, and Hyunchul Shin "Camera and LiDAR-based point painted voxel region-based convolutional neural network for robust 3D object detection," Journal of Electronic Imaging 31(5), 053025 (6 October 2022). https://doi.org/10.1117/1.JEI.31.5.053025
Received: 12 April 2022; Accepted: 20 September 2022; Published: 6 October 2022
Lens.org Logo
CITATIONS
Cited by 1 scholarly publication.
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
LIDAR

Clouds

Image segmentation

Cameras

Convolutional neural networks

Sensors

Image fusion

Back to Top