Investigating the Regularization of Deep Neural Networks for Affect Recognition with Relevance-Guided Local Explanations

Deep neural networks (DNNs) have demonstrated remarkable performance in various computer vision tasks. However, they face challenges that can inhibit their performance and transparency such as the learning of spurious patterns and a lack of explanatory power. This paper addresses these challenges in the domain of affect recognition, particularly for facial expressions. Our first contribution focuses on the integration of domain-specific knowledge into DNNs. To achieve this, we improve on a regularization method that constrains class co-occurrences, thereby outperforming existing state-of-the-art approaches. Our second contribution evaluates the impact of this regularization by employing an adapted explainable AI (XAI) method that incorporates expert knowledge. The results reveal that the regularization term encourages the learning of more generalized features. Consequently, XAI methods enhance the transparency of DNNs, contributing to the development of more reliable AI systems.