Neuro-facial Fusion for Emotion AI: Improved Federated Learning GAN for Collaborative Multimodal Emotion Recognition

다국어 초록 (Multilingual Abstract)

In the context of artificial intelligence technology, an emotion recognition (ER) has numerous roles in human lives. On the other hand, the emotion recognition techniques most currently used perform poorly in recognizing emotions, which limits their wide spread use in practical applications. A Collaborative Multimodal Emotion Recognition through Improved Federated Learning Generative Adversarial Network (MER-IFLGAN) for facial expressions and electro encephalogram (EEG) signals was proposed to reduce this issue. Multi-resolution binarized image feature extraction (MBIFE) was initially used for facial expression feature extraction. The EEG features were extracted using the Dwarf Mongoose Optimization (DMO) algorithm. Finally, IFLGAN completes the Emotion recognition task. The proposed technique was simulated in MATLAB. The proposed technique achieved 25.45% and 19.71% higher accuracy and a 32.01% and 39.11% shorter average processing time compared to the existing models, like EEG based Cross-subject and Cross-modal Model (CSCM) for Multimodal Emotion Recognition (MERCSCM) and Long-Short Term Memory Model (LSTM) for EEG Emotion Recognition (MERLSTM), respectively. The experimental results of the proposed model shows that complementing EEG signals with the features of facial expression could identify four types of emotions: happy, sad, fear, and neutral. Further more, the IFLGAN classifier can enhance the capacity of multimodal emotion recognition.

목차 (Table of Contents)

• Abstract
• 1. Introduction
• 2. Literature Survey
• 3. Proposed Methodology
• 4. Result and Discussion
• 5. Conclusion
• References