Recent developments in the field of affective computing has focused on leveraging physiological signals, such as Electrocardiogram (ECG), to enhance emotion recognition in real-world environments. However, achieving high performance across a wide range of emotion classes presents significant challenges. To address this, we propose a computational Deep Learning framework that utilizes Transfer Learning, allowing for the efficient transfer of knowledge learned from one task to another, for improved performance.

For automatic emotion classification using bio-medical signals, existing methods mostly rely on hand-crafted features such as HRV or PQRST wave complexes of the ECG signals. In this paper, the goal is to develop a Deep Learning model by using bi-channel Electrocardiogram (ECG) data that automatically exploits the time–frequency spectrum of the signal, removing the need for manual feature extraction. The time–frequency RGB color images for ECG signal are extracted using Continuous Wavelet Transform (CWT). The transfer learning of a pre-trained convolution neural network, ConvNeXT is employed to classify these CWT images into emotions on the 2-Dimensional Valence-Arousal space as well as on the 1-Dimensional spaces too. The proposed method is evaluated using the publicly available DREAMER dataset which used the wireless Shimmer2 ECG sensor for data collection.

Evaluation results show that our method outperforms state-of-the-art approaches when classifying on the 2-Dimensional Valence-Arousal space. Emotion elicitation average accuracy of 95.78% was achieved, indicating the effectiveness of our approach in accurately identifying emotions. Reasonable results were also achieved when classifying emotions on a single dimension. Emotion elicitation accuracies of 86.2%, 82.2%, and 83.4% were achieved for the Valence, Arousal & Dominance spaces respectively. These results show that our approach has the potential to contribute to the advancement of affective computing and its applications in various fields.

Keywords: Emotion Recognition (ER), Physiological Signals, Electrocardiogram (ECG), Continuous Wavelet Transform (CWT), Deep Learning, Transfer Learning

Emotions are fundamental in the day to day lives of human beings as they serve a vital role in human cognition ranging from decision making, perceiving things to human interaction and intelligence. They play a major role in communication and can be expressed verbally through emotional vocabulary or non-verbally through gestures or facial expressions. Most of the Human-Computer Interaction (HCI) systems today lack in interpreting the true emotions of the user. The objective of Affective Computing (AC) is to bridge this gap by detecting emotions that occur during Human-Computer Interaction and analyzing emotional responses.

Current Emotion Recognition methods can broadly be classified into 2 main categories: Behavioral patterns and Physiological signals. The former approach classifies emotions based on audio/visual cues such as facial expressions, vocals, and body movements, but can be easily influenced by conscious control of one’s behavior. In contrast, the latter approach, which classifies emotions based on physiological signals such as electrocardiogram (ECG), is less susceptible to conscious control and can more accurately reflect a person’s true emotional state.

The Autonomic Nervous System (ANS) plays a critical role in regulating the heart activity, as it affects the rhythm of cardiac myocyte contractions. The ANS is composed of sympathetic and parasympathetic nervous systems, which have opposing effects on the heart rate. Activation of the cardiac sympathetic nervous system, which is distributed along the atria and ventricles, increases heart rate, while activation of the parasympathetic nervous system reduces the load on the heart. During external mental stimulation, the sympathetic nervous system dominates the regulation of cardiomyocytes. Therefore, the ECG signal is a physiological signal closely related to emotion and is widely used in Emotion Recognition systems

This project aims to fine-tune a simple and robust Pre-Trained Deep Learning Emotion Recognition model that precisely classifies human emotions in a 1 and 2 dimensional space as proposed by James Russell [26], by using Electrocardiogram (ECG) signals. Features of the ECG signal are extracted from the DREAMER database using a Wavelet Scattering algorithm that allows obtaining features of the signal at different time scales, which are then used as inputs for a Deep Learning classifier to evaluate its performance. The proposed model is evaluated against previous State of the Art models to prove the effectiveness and validity of our methodology.