Open Access
Remote Sensing, volume 13, issue 2, pages 326
On the Generalization Ability of Data-Driven Models in the Problem of Total Cloud Cover Retrieval
Publication type: Journal Article
Publication date: 2021-01-19
General Earth and Planetary Sciences
Abstract
Total Cloud Cover (TCC) retrieval from ground-based optical imagery is a problem that has been tackled by several generations of researchers. The number of human-designed algorithms for the estimation of TCC grows every year. However, there has been no considerable progress in terms of quality, mostly due to the lack of systematic approach to the design of the algorithms, to the assessment of their generalization ability, and to the assessment of the TCC retrieval quality. In this study, we discuss the optimization nature of data-driven schemes for TCC retrieval. In order to compare the algorithms, we propose a framework for the assessment of the algorithms’ characteristics. We present several new algorithms that are based on deep learning techniques: A model for outliers filtering, and a few models for TCC retrieval from all-sky imagery. For training and assessment of data-driven algorithms of this study, we present the Dataset of All-Sky Imagery over the Ocean (DASIO) containing over one million all-sky optical images of the visible sky dome taken in various regions of the world ocean. The research campaigns that contributed to the DASIO collection took place in the Atlantic ocean, the Indian ocean, the Red and Mediterranean seas, and the Arctic ocean. Optical imagery collected during these missions are accompanied by standard meteorological observations of cloudiness characteristics made by experienced observers. We assess the generalization ability of the presented models in several scenarios that differ in terms of the regions selected for the train and test subsets. As a result, we demonstrate that our models based on convolutional neural networks deliver a superior quality compared to all previously published approaches. As a key result, we demonstrate a considerable drop in the ability to generalize the training data in the case of a strong covariate shift between the training and test subsets of imagery which may occur in the case of region-aware subsampling.
Citations by journals
|
1
2
|
|
|
Remote Sensing
|
Remote Sensing
2 publications, 18.18%
|
|
Moscow University Physics Bulletin (English Translation of Vestnik Moskovskogo Universiteta, Fizika)
|
Moscow University Physics Bulletin (English Translation of Vestnik Moskovskogo Universiteta, Fizika)
2 publications, 18.18%
|
|
Earth and Space Science
|
Earth and Space Science
1 publication, 9.09%
|
|
Energies
|
Energies
1 publication, 9.09%
|
|
Oceanology
|
Oceanology
1 publication, 9.09%
|
|
Advances in Applied Energy
|
Advances in Applied Energy
1 publication, 9.09%
|
|
Atmospheric Measurement Techniques
|
Atmospheric Measurement Techniques
1 publication, 9.09%
|
|
AIP Conference Proceedings
|
AIP Conference Proceedings
1 publication, 9.09%
|
|
Izvestiya - Atmospheric and Oceanic Physics
|
Izvestiya - Atmospheric and Oceanic Physics
1 publication, 9.09%
|
|
1
2
|
Citations by publishers
|
1
2
3
|
|
|
Multidisciplinary Digital Publishing Institute (MDPI)
|
Multidisciplinary Digital Publishing Institute (MDPI)
3 publications, 27.27%
|
|
Pleiades Publishing
|
Pleiades Publishing
2 publications, 18.18%
|
|
Allerton Press
|
Allerton Press, 2, 18.18%
Allerton Press
2 publications, 18.18%
|
|
Wiley
|
Wiley
1 publication, 9.09%
|
|
Elsevier
|
Elsevier
1 publication, 9.09%
|
|
Copernicus
|
Copernicus
1 publication, 9.09%
|
|
American Institute of Physics (AIP)
|
American Institute of Physics (AIP)
1 publication, 9.09%
|
|
1
2
3
|
- We do not take into account publications that without a DOI.
- Statistics recalculated only for publications connected to researchers, organizations and labs registered on the platform.
- Statistics recalculated weekly.
{"yearsCitations":{"type":"bar","data":{"show":true,"labels":[2022,2023,2024],"ids":[0,0,0],"codes":[0,0,0],"imageUrls":["","",""],"datasets":[{"label":"Citations number","data":[3,7,1],"backgroundColor":["#3B82F6","#3B82F6","#3B82F6"],"percentage":["27.27","63.64","9.09"],"barThickness":null}]},"options":{"indexAxis":"x","maintainAspectRatio":true,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":1,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Citations per year","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}},"journals":{"type":"bar","data":{"show":true,"labels":["Remote Sensing","Moscow University Physics Bulletin (English Translation of Vestnik Moskovskogo Universiteta, Fizika)","Earth and Space Science","Energies","Oceanology","Advances in Applied Energy","Atmospheric Measurement Techniques","AIP Conference Proceedings","Izvestiya - Atmospheric and Oceanic Physics"],"ids":[22827,3806,16245,23828,12808,35440,22345,21274,19686],"codes":[0,0,0,0,0,0,0,0,0],"imageUrls":["\/storage\/images\/resized\/MjH1ITP7lMYGxeqUZfkt2BnVLgjkk413jwBV97XX_medium.webp","\/storage\/images\/resized\/oZgeErrVFhuDksyqFURLvYS1wtVSBWczh001igGo_medium.webp","\/storage\/images\/resized\/bRyGpdm98BkAUYiK1YFNpl5Z7hPu6Gd87gbIeuG3_medium.webp","\/storage\/images\/resized\/MjH1ITP7lMYGxeqUZfkt2BnVLgjkk413jwBV97XX_medium.webp","\/storage\/images\/resized\/oZgeErrVFhuDksyqFURLvYS1wtVSBWczh001igGo_medium.webp","\/storage\/images\/resized\/GDnYOu1UpMMfMMRV6Aqle4H0YLLsraeD9IP9qScG_medium.webp","\/storage\/images\/resized\/cxBZptDQQvWgQeXj5GoTXDfgUEoGVvFgpDRctnDr_medium.webp","\/storage\/images\/resized\/ARM4e6URKRsbRZvIF0vFis9DjxGloBjnBYJXbHmZ_medium.webp","\/storage\/images\/resized\/oZgeErrVFhuDksyqFURLvYS1wtVSBWczh001igGo_medium.webp"],"datasets":[{"label":"","data":[2,2,1,1,1,1,1,1,1],"backgroundColor":["#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6"],"percentage":[18.18,18.18,9.09,9.09,9.09,9.09,9.09,9.09,9.09],"barThickness":13}]},"options":{"indexAxis":"y","maintainAspectRatio":false,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":null,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Journals","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}},"publishers":{"type":"bar","data":{"show":true,"labels":["Multidisciplinary Digital Publishing Institute (MDPI)","Pleiades Publishing","Allerton Press","Wiley","Elsevier","Copernicus","American Institute of Physics (AIP)"],"ids":[202,101,6983,11,17,3207,250],"codes":[0,0,0,0,0,0,0],"imageUrls":["\/storage\/images\/resized\/MjH1ITP7lMYGxeqUZfkt2BnVLgjkk413jwBV97XX_medium.webp","\/storage\/images\/resized\/oZgeErrVFhuDksyqFURLvYS1wtVSBWczh001igGo_medium.webp","","\/storage\/images\/resized\/bRyGpdm98BkAUYiK1YFNpl5Z7hPu6Gd87gbIeuG3_medium.webp","\/storage\/images\/resized\/GDnYOu1UpMMfMMRV6Aqle4H0YLLsraeD9IP9qScG_medium.webp","\/storage\/images\/resized\/cxBZptDQQvWgQeXj5GoTXDfgUEoGVvFgpDRctnDr_medium.webp","\/storage\/images\/resized\/ARM4e6URKRsbRZvIF0vFis9DjxGloBjnBYJXbHmZ_medium.webp"],"datasets":[{"label":"","data":[3,2,2,1,1,1,1],"backgroundColor":["#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6"],"percentage":[27.27,18.18,18.18,9.09,9.09,9.09,9.09],"barThickness":13}]},"options":{"indexAxis":"y","maintainAspectRatio":false,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":null,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Publishers","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}}}
Metrics
Cite this
GOST |
RIS |
BibTex |
MLA
Cite this
GOST
Copy
Krinitskiy M. et al. On the Generalization Ability of Data-Driven Models in the Problem of Total Cloud Cover Retrieval // Remote Sensing. 2021. Vol. 13. No. 2. p. 326.
GOST all authors (up to 50)
Copy
Krinitskiy M., Aleksandrova M., Verezemskaya P., Gulev S., Sinitsyn A., Kovaleva N., Gavrikov A. On the Generalization Ability of Data-Driven Models in the Problem of Total Cloud Cover Retrieval // Remote Sensing. 2021. Vol. 13. No. 2. p. 326.
Cite this
RIS
Copy
TY - JOUR
DO - 10.3390/rs13020326
UR - https://doi.org/10.3390%2Frs13020326
TI - On the Generalization Ability of Data-Driven Models in the Problem of Total Cloud Cover Retrieval
T2 - Remote Sensing
AU - Aleksandrova, Marina
AU - Gulev, Sergey
AU - Sinitsyn, Alexey
AU - Kovaleva, Nadezhda
AU - Gavrikov, Alexander
AU - Krinitskiy, Mikhail
AU - Verezemskaya, Polina
PY - 2021
DA - 2021/01/19 00:00:00
PB - Multidisciplinary Digital Publishing Institute (MDPI)
SP - 326
IS - 2
VL - 13
SN - 2072-4292
ER -
Cite this
BibTex
Copy
@article{2021_Krinitskiy,
author = {Marina Aleksandrova and Sergey Gulev and Alexey Sinitsyn and Nadezhda Kovaleva and Alexander Gavrikov and Mikhail Krinitskiy and Polina Verezemskaya},
title = {On the Generalization Ability of Data-Driven Models in the Problem of Total Cloud Cover Retrieval},
journal = {Remote Sensing},
year = {2021},
volume = {13},
publisher = {Multidisciplinary Digital Publishing Institute (MDPI)},
month = {jan},
url = {https://doi.org/10.3390%2Frs13020326},
number = {2},
pages = {326},
doi = {10.3390/rs13020326}
}
Cite this
MLA
Copy
Krinitskiy, Mikhail, et al. “On the Generalization Ability of Data-Driven Models in the Problem of Total Cloud Cover Retrieval.” Remote Sensing, vol. 13, no. 2, Jan. 2021, p. 326. https://doi.org/10.3390%2Frs13020326.