Online segmentation of street trees from mobile laser scanning data via deep learning image instance segmentation

A mobile laser scanning (MLS) system can efficiently collect the vertical structure information of street trees through light detection and ranging (LiDAR) movement. Segmenting street tree instances from street MLS data is the primary step in conducting street tree inventories at the single-tree scale, and existing methods are limited by the high computational complexity of three-dimensional (3D) point cloud processing, resulting in low speed and not supporting real-time applications. This paper proposes an efficient and effective online framework for segmenting street trees from street MLS data via deep learning image instance segmentation. First, the point cloud data collected in real time by LiDAR are stored in a first-in-first-out (FIFO) buffer. When the number of new LiDAR frames reaches the set segmentation step, the point cloud data in the FIFO buffer are converted into a three-channel image using point-pixel mapping. Next, a trained high-performance YOLOv8 segmentation model is used to segment an image instance of the street tree quickly. Then, a point cloud instance of the street tree is accurately segmented through optimization at the pixel and point levels. Finally, the segmentation step is adaptively adjusted on the basis of the street tree image instance segmentation result to improve the online processing efficiency. During the experiment, an MLS system equipped with a UTM-30LX-EW sensor is used to collect one-sided street point cloud data containing 149 street trees and other objects, such as landscape plants, buildings, viaducts, lanes, sidewalks, street lights, traffic signs, trash cans, baffles, fences, cars, motorcycles, and pedestrians. Eight YOLOv8 models with different target image sizes are compared, and the 416 × 320 model is selected to balance segmentation accuracy and speed, with an AP90 of 0.9950 and a segmentation time per image of 16.81 ms. Through pixel- and point-level optimization, the F1 score for online segmentation of street tree point cloud instances reaches 0.9974. The online segmentation time per frame is 2.66 ms, which is less than the 25 ms LiDAR frame period, and real-time segmentation is achieved. In the comparison experiments with state-of-the-art methods, the proposed method outperforms the other two methods in both accuracy and speed. The experimental results show that the proposed framework can quickly and accurately segment street tree point cloud instances from online MLS data.