Conditioned deep feature consistent variational autoencoder for simulating realistic sonar images

Abstract

Multibeam imaging sonar is one of the primary sensors for underwater navigation with uncrewed underwater vehicles (UUVs) due to the robustness to turbidity and variable lighting conditions that limit the applicability of standard cameras. However, the operating principles and noise models of real sensors make imaging sonar challenging to accurately simulate, and acquiring real images experimentally is difficult and costly. This paper presents an approach for transforming a synthetically generated input image into the textural domain of real sonar images using a variational autoencoder (VAE) with a modified loss function. This allows us to generate realistic sonar images of simulated scenarios emulating the texture of real acoustic images. As a result, large datasets can be created from a relatively small amount of real images, which can be later used in many downstream applications, ranging from evaluating data association algorithms to deep learning. The method was evaluated using an isolated real and simulated dataset that trained a separate convolutional neural network (CNN) to discern between images in the sonar domain and simulated images. The VAE has several advantages over a compared Cycle Consistent Generative Adversarial Network (CycleGAN) approach, including more texturally accurate generated images, and allowing for more variation in generated images.

Miguel Castillón

Autonomous Navigation Lead

My main research interests include exploiting sensor perception for autonomous navigation and manipulation in challenging scenarios.