Road surface damage detection based on hierarchical architecture using lightweight auto-encoder network

• Hierarchical neural network structure to extract various features of road distress. • Training and prediction method with multiple loss functions and weighted soft voting. • Computationally efficient road damage detection with high recognition performance. • Multiple road damage detection with an accuracy of 81.62% m-IoU and 79.33% F1. • Real-time performance algorithm for personal mobility vehicle safety. In this paper, we propose a novel neural network structure and training and prediction methods. We propose a novel deep neural network algorithm to detect road surface damage conditions for establishing a safe road environment. We secure 1300 training and 400 testing images to train the neural network; the images contain multiple types of road distress. The proposed algorithm is compared with nine deep learning models from various fields. Comparison results indicate that the proposed algorithm outperforms all others with a pixel accuracy of 97.61%, F1 score of 79.33%, mean intersection over union of 81.62%, and frequency-weighted intersection over union of 95.64%; in addition, it requires only 3.56 M parameters. In the future, the results of this study are expected to play an important role in ensuring safe driving by efficiently detecting poor road conditions.