The Usage of Grayscale or Color Images for Facial Expression Recognition with Deep Neural Networks

The paper describes usage of modern deep neural network architectures such as ResNet, DenseNet and Xception for the classification of facial expressions on color and grayscale images. Each image may contain one of eight facial expression categories: “Neutral”, “Happiness”, “Sadness”, “Surprise”, “Fear”, “Disgust”, “Anger”, “Contempt”. As the dataset was used AffectNet. The most accurate architecture is Xception. It gave classification accuracy on training sample 97.65%, on cleaned testing sample 57.48% and top-2 accuracy on cleaned testing sample 76.70%. The category “Contempt” is worst recognized by all the types of neural networks considered, which indicates its ambiguity and similarity with other types of facial expressions. Experimental results show that for the considered task it does not matter, the color or grayscale image is fed to the input of the algorithm. This fact can save a significant amount of memory when storing data sets and training neural networks. The computing experiments was performed using graphics processor using NVidia CUDA technology with Keras and Tensorflow deep learning frameworks. It showed that the average processing time of one image varies from 4 ms to 30 ms for different architectures. Obtained results can be used in software for neural network training for face recognition systems.