Visual Cross-Modal Mapping, Labeling and Localization

Elhousni, Mahdi

Etd

Visual Cross-Modal Mapping, Labeling and Localization

Public Deposited

Mapping, labeling and localization are central modules to most autonomous robot platforms. Not only do they make it possible for the robot to understand its surrounding environment and its location in it, it can also improve drastically the accuracy of other downstream tasks such as behavioral planning or dynamic object detection and tracking. While it is possible to achieve reasonable results with traditional dynamic sensors such as IMUs and GPS, modern robotics systems have shown that visual based sensors (mainly laser or camera based) are extremely well suited for these tasks, since the final localization results returned by such sensors do not only depend on the robot itself, but also on its surrounding environment. While both types of sensors possess numerous qualities that result into a good performance, they also suffer from some limitations. Such limitation can sometimes be overcome by using cross-modal approaches. This thesis shows how we can take advantage of such methods. First, we start by exploring the cross-modal mapping task for UAVs, where we use deep learning-based height prediction to construct 3D point cloud maps using monocular images only. Next, we explore the cross-modal labeling task, by using camera data to predict relevant road features, before processing them to label pre-built point cloud maps. Finally, we tackle the case of cross-modal vehicle localization of point clouds when 3D maps are unavailable, by proposing a constrained method able to localize LiDAR scans in OpenStreetMaps.

Creator