Hyperspectral Image-Based Identification of Maritime Objects Using Convolutional Neural Networks and Classifier Models

Liu C., Zhang Y., Shen J., Liu F.

Infrared cameras form images by capturing the thermal radiation emitted by objects in the infrared spectrum, making them complex sensors widely used in maritime surveillance. However, the broad spectral range of the infrared band makes it susceptible to environmental interference, which can reduce the contrast between the target and the background. As a result, detecting infrared targets in complex marine environments remains challenging. This paper presents a novel and enhanced detection model developed from the real-time detection transformer (RT-DETR), which is designated as MAFF-DETR. The model incorporates a novel backbone by integrating CSP and parallelized patch-aware attention to enhance sensitivity to infrared imagery. Additionally, a channel attention module is employed during feature selection, leveraging high-level features to filter low-level information and enabling efficient multi-level fusion. The model’s target detection performance on resource-constrained devices is further enhanced by incorporating advanced techniques such as group convolution and ShuffleNetV2. The experimental results show that, although the enhanced RT-DETR algorithm still experiences missed detections under severe object occlusion, it has significantly improved overall performance, including a 1.7% increase in mAP, a reduction in 4.3 M parameters, and a 5.8 GFLOPs decrease in computational complexity. It can be widely applied to tasks such as coastline monitoring and maritime search and rescue.

Zhan C., Bai K., Tu B., Zhang W.

Offshore oil spills have the potential to inflict substantial ecological damage, underscoring the critical importance of timely offshore oil spill detection and remediation. At present, offshore oil spill detection typically combines hyperspectral imaging with deep learning techniques. While these methodologies have made significant advancements, they prove inadequate in scenarios requiring real-time detection due to limited model detection speeds. To address this challenge, a method for detecting oil spill areas is introduced, combining convolutional neural networks (CNNs) with the DBSCAN clustering algorithm. This method aims to enhance the efficiency of oil spill area detection in real-time scenarios, providing a potential solution to the limitations posed by the intricate structures of existing models. The proposed method includes a pre-feature selection process applied to the spectral data, followed by pixel classification using a convolutional neural network (CNN) model. Subsequently, the DBSCAN algorithm is employed to segment oil spill areas from the classification results. To validate our proposed method, we simulate an offshore oil spill environment in the laboratory, utilizing a hyperspectral sensing device to collect data and create a dataset. We then compare our method with three other models—DRSNet, CNN-Visual Transformer, and GCN—conducting a comprehensive analysis to evaluate the advantages and limitations of each model.

Park J., Park K., Kim T., Oh S., Lee M.

Over the past decades, maritime accidents have been increasing due to the rise in maritime transportation and ship traffic. While detecting accident-prone vessels is crucial, it is equally important to identify individuals in distress and small floating objects. Real-time monitoring and wide-area high-resolution observations enabled by aerial remote sensing have proven effective in maritime detection. In this study, we developed a technology for detecting small objects by conducting two aerial experiments targeting various objects, including ships, mannequins (human-shaped objects), and maritime safety equipment floating in coastal areas, thereby acquiring hyperspectral image data. By utilizing the hyperspectral data, we detected the pixels corresponding to the edges of ships and employed an ellipse fitting approach to identify the vessels, achieving a length error of 0.44 meters. Additionally, we detected small floating objects based on a spectral database using spectral matching. The N-finder algorithm (N-FINDR) spectral unmixing algorithm was applied to detect lifebuoys, buoyant apparatus, and mannequins, resulting in relatively small length errors ranging from 0.08 to 0.17 meters. As satellite hyperspectral sensors continue to advance significantly, it is expected that this study will contribute to future research in the field of detecting small objects and maritime surveillance.

Xia Z., Ma K., Cheng S., Blackburn T., Peng Z., Zhu K., Zhang W., Xiao D., Knowles A., Arcucci R.

The performance of advanced materials for extreme environments is underpinned by their microstructure, such as the size and distribution of nano- to micro-sized reinforcing phase(s). Chromium-based superalloys are a recently...

Zhu K., Cheng S., Kovalchuk N., Simmons M.J., Guo Y., Matar O., Arcucci R.

Predicting drop coalescence based on process parameters is crucial for experiment design in chemical engineering. However, predictive models can suffer from the lack of training data and more importantly, the...

Yang Q., Li Z., Li J., An H., Wu J., Pi Y., Yang J.

Due to the advantages of flexible configuration, bistatic synthetic aperture radar (BiSAR) has the ability to effectively observe from various visual angles, such as forward view area and squint area, and has good anti-jamming characteristics. It can be applied to the surveillance of ship targets on the sea and is gradually gaining an increasing amount of attention. However, for ship targets with complex motions on the sea surface, such as maneuvering targets or ship targets under high sea conditions, the high-order Doppler frequency of the scattering points is always spatial variation (related to the spatial position of scattering points), which poses a considerable challenge for the imaging of maritime ship targets in BiSAR. To resolve this problem, a BiSAR maritime ship target imaging algorithm based on cubic phase time-scaled transformation is proposed in this paper. First, through pre-processing of echo such as Doppler prefiltering and keystone transform, the translation compensation of the BiSAR maritime ship target is completed, and the scattering point energy is corrected to within one range unit. Then, the azimuth signal is modeled as a multi-component cubic phase signal. Based on the proposed cubic phase time-scaled transformation, the Doppler centroid, frequency rate, and third-order frequency of scattering points are estimated. Eventually, the BiSAR imaging of maritime ship targets is realized. This algorithm has excellent noise immunity and low cross-terms. The simulation leads to the verification of the validity of the proposed algorithm.

Li Z., Chen J., Xiong Y., Yu H., Zhang H., Gao B.

Ship detection and management in coastal regions are challenging tasks due to the complex appearances of ships and the background. For further applications in the context of fisheries monitoring and vessel traffic services, a single-channel synthetic aperture radar (SAR) is mounted on a number of maneuvering and inexpensive rotor platforms, which are utilized according to the consideration of flexible observation, cost savings, weight, and space constraints. In this paper, a hierarchical scheme of ship detection, ship imaging, and classification is proposed. It mainly includes three parts. First, a mixture statistical model of semi-parametric K-lognormal distribution based on adaptive background windows with a constant false alarm rate (CFAR) is proposed for ship prescreening in SAR imagery. Then, the discrimination stage, combined with ship imaging via the difference between the true ship targets and the false ones in the aspects of micro-Doppler motion properties, is performed. Finally, the simulation and field data processing results are presented to validate the proposed scheme.

Taggio N., Aiello A., Ceriola G., Kremezi M., Kristollari V., Kolokoussis P., Karathanassi V., Barbone E.

A significant amount of the produced solid waste reaching the oceans is made of plastics. The amount of plastic debris in the ocean and coastal areas is steadily increasing and is now a major global environmental issue. The monitoring of marine plastic litter, ground-based monitoring systems and/or field campaigns are time-consuming, expensive, require great organisational efforts, and provide very limited information in terms of the spatial and temporal dynamics of marine debris. Earth Observation (EO) by satellite can contribute significantly to marine plastic litter detection. In 2019, a new hyperspectral satellite, called PRISMA, was launched by the Italian Space Agency. The high spectral resolution of PRISMA may allow for better detection of floating plastic materials. At the same time, Machine Learning (ML) algorithms have the potential to find hidden patterns and identify complex relations among data and are increasingly employed in EO. This paper presents the development of a new method of identifying floating plastic objects in coastal areas by exploiting pan-sharpened hyperspectral PRISMA data, based on the combination of unsupervised and supervised ML algorithms. The study consisted of a configuration phase, during which the algorithms were trained in a fully controlled test, and a validation phase, in which the pre-trained algorithms were applied to satellite data collected at different sites and in different periods of the year. Despite the limited input data, results suggest that the tested ML approach, applied to pan-sharpened PRISMA data, can effectively recognise floating objects and plastic targets. The study indicates that increasing input datasets can help achieve higher-quality results.

Kruse R., Mostaghim S., Borgelt C., Braune C., Steinbrecher M.

Li H., Liao G., Xu J., Lan L.

In this paper, a joint maritime moving target detection and imaging approach, referred to as the fast inverse synthetic aperture radar (ISAR) imaging approach, based on the multi-resolution space−time adaptive processing (STAP), is proposed to improve the target detection performance and the target imaging efficiency in an airborne radar system. In the target detection stage, the sub-band STAP is introduced to improve the robustness of clutter suppression and to enhance the target output power with the decreased range resolution, by which the coarse estimation of target range-Doppler (R-D) location is obtained as the prior knowledge. In the following target imaging stage, the ISAR imaging is applied in the localized R-D zone surrounding with the target location. However, it is difficult to directly apply ISAR imaging with the conventional R-D algorithm because the slow-moving maritime target cannot be separated from the clutter interference in the Doppler frequency dimension. In this regard, the full-band STAP is applied in the R-D two-dimensional frequency domain for the simultaneous clutter suppression and high-resolution ISAR imaging, in which the envelope alignment and phase compensation are achieved by adaptive match filtering with the target Doppler frequency coarse estimation. Moreover, the reduced-dimension STAP applied in the target-surrounded localized Doppler frequency zone gives facilities for alleviating the computation burden. Simulation results corroborate the effectiveness of the proposed method.

Jiang Z., Zhang J., Ma Y., Mao X.

Marine oil spills can damage marine ecosystems, economic development, and human health. It is important to accurately identify the type of oil spills and detect the thickness of oil films on the sea surface to obtain the amount of oil spill for on-site emergency responses and scientific decision-making. Optical remote sensing is an important method for marine oil-spill detection and identification. In this study, hyperspectral images of five types of oil spills were obtained using unmanned aerial vehicles (UAV). To address the poor spectral separability between different types of light oils and weak spectral differences in heavy oils with different thicknesses, we propose the adaptive long-term moment estimation (ALTME) optimizer, which cumulatively learns the spectral characteristics and then builds a marine oil-spill detection model based on a one-dimensional convolutional neural network. The results of the detection experiment show that the ALTME optimizer can store in memory multiple batches of long-term oil-spill spectral information, accurately identify the type of oil spills, and detect different thicknesses of oil films. The overall detection accuracy is larger than 98.09%, and the Kappa coefficient is larger than 0.970. The F1-score for the recognition of light-oil types is larger than 0.971, and the F1-score for detecting films of heavy oils with different film thicknesses is larger than 0.980. The proposed optimizer also performs well on a public hyperspectral dataset. We further carried out a feasibility study on oil-spill detection using UAV thermal infrared remote sensing technology, and the results show its potential for oil-spill detection in strong sunlight.

Soldi G., Gaglione D., Forti N., Simone A.D., Daffina F.C., Bottini G., Quattrociocchi D., Millefiori L.M., Braca P., Carniel S., Willett P., Iodice A., Riccio D., Farina A.

Maritime surveillance (MS) is crucial for search and rescue operations, fishery monitoring, pollution control, law enforcement, migration monitoring, and national security policies. Since the early days of seafaring, MS has been a critical task for providing security in human coexistence. Several generations of sensors providing detailed maritime information have become available for large offshore areas in real time: maritime radar sensors in the 1950s and the automatic identification system (AIS) in the 1990s among them. However, ground-based maritime radars and AIS data do not always provide a comprehensive and seamless coverage of the entire maritime space. Therefore, the exploitation of space-based sensor technologies installed on satellites orbiting around the earth, such as satellite AIS data, synthetic aperture radar, optical sensors, and global navigation satellite systems reflectometry, becomes crucial for MS and to complement the existing terrestrial technologies. In the first part of this work, we provide an overview of the main available space-based sensors technologies and present the advantages and limitations of each technology in the scope of MS. The second part, related to artificial intelligence, signal processing, and data fusion techniques, is provided in a companion paper, titled: "Space-Based Global Maritime Surveillance. Part II: Artificial Intelligence and Data Fusion Techniques".

Freitas S., Silva H., Silva E.

This paper addresses the development of a remote hyperspectral imaging system for detection and characterization of marine litter concentrations in an oceanic environment. The work performed in this paper is the following: (i) an in-situ characterization was conducted in an outdoor laboratory environment with the hyperspectral imaging system to obtain the spatial and spectral response of a batch of marine litter samples; (ii) a real dataset hyperspectral image acquisition was performed using manned and unmanned aerial platforms, of artificial targets composed of the material analyzed in the laboratory; (iii) comparison of the results (spatial and spectral response) obtained in laboratory conditions with the remote observation data acquired during the dataset flights; (iv) implementation of two different supervised machine learning methods, namely Random Forest (RF) and Support Vector Machines (SVM), for marine litter artificial target detection based on previous training. Obtained results show a marine litter automated detection capability with a 70–80% precision rate of detection in all three targets, compared to ground-truth pixels, as well as recall rates over 50%.

Qiao D., Liu G., Lv T., Li W., Zhang J.

The primary task of marine surveillance is to construct a perfect marine situational awareness (MSA) system that serves to safeguard national maritime rights and interests and to maintain blue homeland security. Progress in maritime wireless communication, developments in artificial intelligence, and automation of marine turbines together imply that intelligent shipping is inevitable in future global shipping. Computer vision-based situational awareness provides visual semantic information to human beings that approximates eyesight, which makes it likely to be widely used in the field of intelligent marine transportation. We describe how we combined the visual perception tasks required for marine surveillance with those required for intelligent ship navigation to form a marine computer vision-based situational awareness complex and investigated the key technologies they have in common. Deep learning was a prerequisite activity. We summarize the progress made in four aspects of current research: full scene parsing of an image, target vessel re-identification, target vessel tracking, and multimodal data fusion with data from visual sensors. The paper gives a summary of research to date to provide background for this work and presents brief analyses of existing problems, outlines some state-of-the-art approaches, reviews available mainstream datasets, and indicates the likely direction of future research and development. As far as we know, this paper is the first review of research into the use of deep learning in situational awareness of the ocean surface. It provides a firm foundation for further investigation by researchers in related fields.

Young K.S., Pradhanang S.M.

Submarine Groundwater Discharge (SGD) represents a significant mode of chemical transport to water bodies, making it an important flux to understand. Small Unmanned Aircraft Systems-deployed thermal infrared sensors (sUAS-TIR) provide a financially and logistically inexpensive means of identifying SGD source zones and quantifying SGD thermal infrared (TIR) plume areas over regional scales at high spatial resolutions. sUAS-TIR additionally offers the unique capability of high temporal resolution measurements of SGD. As a developing science application, the use of sUAS-TIR to image SGD requires substantial background knowledge. We present a proposed methodological construct for implementing a sUAS-TIR program for SGD-TIR data gathering, with applications extending to other research fields that can benefit from airborne TIR. Several studies have used airborne TIR in combination with empirical SGD flux measurements to quantify SGD, reporting a consistently strong regression between SGD flux and SGD TIR plume area. We additionally discuss novel research opportunities for sUAS-TIR technologies, as applied to SGD flux. The combination of high spatial and temporal resolution capabilities, at relatively low costs, make sUAS-TIR a promising new technology to overcome the scaling challenges presented by empirical studies and modeling of SGD fluxes, and advance our understanding of the controls on SGD fluxes.

Chen X., Li C., Wang H., Tai Y., Wang J., Migniot C.

Predicting the uncertain distribution of underwater acoustic fields, influenced by dynamic oceanic parameters, is critical for acoustic applications that rely on sound field characteristics to generate predictions. Traditional methods, such as the Monte Carlo method, are computationally intensive and thus unsuitable for applications requiring high real-time performance and flexibility. Current machine learning methods excel at improving computational efficiency but face limitations in predictive performance, especially in shadow areas. In response, a machine learning method is proposed in this paper that balances accuracy and efficiency for predicting uncertainties in deep ocean acoustics by decoupling the scene representation into two components: (a) a local radiance model related to environmental factors, and (b) a global representation of the overall scene context. Specifically, the internal relationships within the local radiance are first exploited, aiming to capture fine-grained details within the acoustic field. Subsequently, local clues are combined with receiver location information for joint learning. To verify the effectiveness of the proposed approach, a dataset of historical oceanographic data has been compiled. Extensive experiments validate the efficiency compared to traditional Monte Carlo techniques and the superior accuracy compared to existing learning method.