Empirical and modeling study of emotional state dynamics in social videogame paradigms

The objective of this work was to study the dynamics of human emotional states in the process of social interaction in a virtual environment. The previously developed for this purpose prototypes of the virtual actor (NPC) and its virtual environment simulator “Teleport” underwent significant re-design and modification. The experimental platform was re-implemented and used in experiments with college student participants, combining electromyography, emotion recognition in facial recordings and model-based game log analysis in a social videogame paradigm. Participants interacted with two virtual actors implemented based on the eBICA cognitive architecture (Samsonovich, 2013, 2018). Positive correlations were found between eBICA model predictions and participant affects extracted from their facial expressions and facial muscle activity. Affective dynamics of social phenomena, such as the establishment of partnership or an act of betrayal, were characterized and found consistent with the model predictions. Other findings include a gradually developing emotional reaction, possibly due to the integration of appraisals of game events. Overall, obtained results confirm the eBICA model, suggesting its further extension and refinement.