Unsupervised learning techniques for finding missing bolts in railways combining 3D data, deep learning, and weighted content loss

One of the most pressing matters in the railway transportation framework is ensuring the integrity of the infrastructure. To this end, the timely detection of potentially dangerous anomalies on every structural element, from the track to the bridge, is mandatory and must be achieved via continuous monitoring. One critical aspect is related to the bolts used to fasten the sleepers to the track; the lack of which may lead to track deformations and, ultimately, potentially catastrophic failures. To deal with this issue, this study proposes a framework for assessing the integrity of the railway track and detecting the lack of fasteners using deep autoencoders and three-dimensional data. Specifically, the study starts from the 3D imagery acquired using stereo cameras mounted underneath a diagnostic train. These three-dimensional maps provide relevant information on the 3D structure of the railway along the z-axis and, if projected on a bi-dimensional image, can represent the structure of the fastener. Imagery obtained in this way was fed to an autoencoder, which used a custom loss function designed to augment the traditional reconstruction error with a content-related term computed starting from a deep convolutional neural network specifically trained to extract meaningful features from the original images. The content loss allowed for improvement in the overall performance of the autoencoder in anomaly detection, providing an increment of 1.31% in terms of F1 score and 2.37% in terms of accuracy, thus achieving robust identification of anomalies and reducing the overall amount of false negatives that may arise during inspection, ultimately contributing to the advancement in quality control processes for critical infrastructure monitoring.