Smartphone-based road manhole cover detection and classification

Road surface condition detection is an important application for many intelligent transportation systems (ITSs). A manhole cover depression is one of the common factors affecting road conditions. Smartphones are equipped with different sensors, which can be used to collect image data and inertial data. A new large-scale manhole cover detection dataset is developed by using smartphones to collect road image data, and a hierarchical classification method based on the convolutional neural network is proposed in this paper. The proposed method first coarsely classifies the images into nonrainy and rainy types and then performs manhole cover detections based on the coarse classification results. As a result, the proposed method achieves an accuracy of approximately 86.3% for road manhole cover detection. Based on the observation that different degrees of manhole cover subsidence produce different degrees of inertial sensor data, this paper used a machine learning method, which can automatically classify the detected manhole covers into different degrees of subsidence, namely good, average, and poor. The average recalls, average precisions, and average F1-measures achieve approximately 87.3%, 86.9%, and 87.2% accuracy, respectively. The results show that the proposed approach can effectively detect manhole covers in different weather and road conditions, which can effectively reduce the cost of road manhole cover data collection and detection, providing a new method for road manhole cover detection. • Road manhole cover detection and classification using smartphones. • The hierarchical model improves the image detection accuracy of manhole covers. • A method to classify the subsidence of road manhole covers based on inertial data. • The method can detect manhole cover subsidence in different weather conditions.