Underwater Vision-Based Gesture Recognition: A Robustness Validation for Safe Human-Robot Interaction

Underwater robotics requires very reliable and safe operations. This holds especially for missions in cooperation with divers who are - despite the significant advancements of marine robotics in recent years - still essential for many underwater operations. Possible application cases of underwater human-robot collaboration include marine science, archeology, oil- and gas production (OGP), handling of unexploded ordnance (UXO), e.g., from WWII ammunition dumped in the seas, or inspection and maintenance of marine infrastructure like pipelines, harbors, or renewable energy installations - to name just a few examples. We present a fully integrated approach to Underwater Human Robot Interaction (U-HRI) in form of a front-end for gesture recognition combined with a back-end with a full language interpreter. The gesture-based language is derived from the existing standard gestures for communication between human divers. It enables a diver to issue single commands as well as complex mission specifications to an Autonomous Underwater Vehicle (AUV) as demonstrated in several field trials. The gesture recognition is an essential component of the overall approach. It requires high reliability under the challenging conditions of the underwater domain. There is especially a high amount of variation in visual data due to various effects in the underwater image formation. We hence investigate in this article different Machine Learning (ML) methods for robust diver gesture recognition. This includes a classical ML approach and four state-of-the-art Deep Learning (DL) methods. Furthermore, we introduce a physically realistic way to use range information for adding underwater haze to produce meaningful additional data from existing real-world data. This can be of interest for creating evaluation data for underwater perception in general or to produce additional training data for ML-based approaches.