doi: 10.56294/dm2023162
ORIGINAL
Optimizing Emotion Recognition of Non-Intrusive E-Walking Dataset
Optimización del reconocimiento de emociones en un conjunto de datos no intrusivos de e-walking
1Mody University of Science & Technology, Computer Science & Engineering. Laxmangarh, India.
2Mody University of Science & Technology, Computer Science & Engineering. Laxmangarh, India.
Cite as: Jain P, Maan V. Optimizing Emotion Recognition of Non-Intrusive E-Walking Dataset. Data and Metadata. 2023; 2:162. https://doi.org/10.56294/dm2023162
Submitted: 06-09-2023 Revised: 02-11-2023 Accepted: 29-12-2023 Published: 30-12-2023
Editor: Prof.
Dr. Javier González Argote 
ABSTRACT
Emotion recognition being a complex task because of its valuable usages in critical fields like Robotics, human-computer interaction and mental health has recently gathered huge attention. The selection and optimization of suitable feature sets that can accurately capture the underlying emotional states is one of the critical challenges in Emotion Recognition. Metaheuristic optimization techniques have shown promise in addressing this challenge by efficiently exploring the large and complex feature space. This research paper proposes a novel framework for emotion recognition that uses metaheuristic optimization. The key idea behind metaheuristic optimization is to explore the search space in an intelligent way, by generating candidate solutions and iteratively improving them until an optimal or near-optimal solution is found. The accuracy & robustness of emotion identification systems can be enhanced by optimizing the metaheuristic optimization. The major contribution of this research is to develop a Chiropteran Mahi Metaheuristic optimization which emphasizes the weights updating in the classifier for improving the accuracy of the proposed system.
Keywords: Metaheuristic Optimization; Emotion Recognition (ER); Chiropteran Mahi Metaheuristic; E-Walking.
RESUMEN
El reconocimiento de emociones es una tarea compleja debido a sus valiosos usos en campos críticos como la robótica, la interacción persona-ordenador y la salud mental. La selección y optimización de conjuntos de características adecuadas que puedan capturar con precisión los estados emocionales subyacentes es uno de los retos críticos en el reconocimiento de emociones. Las técnicas de optimización metaheurística se han mostrado prometedoras para abordar este reto explorando de forma eficiente el amplio y complejo espacio de características. Este trabajo de investigación propone un nuevo marco para el reconocimiento de emociones que utiliza la optimización metaheurística. La idea clave de la optimización metaheurística es explorar el espacio de búsqueda de forma inteligente, generando soluciones candidatas y mejorándolas iterativamente hasta encontrar una solución óptima o casi óptima. La precisión y robustez de los sistemas de identificación de emociones pueden mejorarse mediante la optimización metaheurística. La principal contribución de esta investigación es el desarrollo de una optimización metaheurística Chiropteran Mahi que hace hincapié en la actualización de los pesos en el clasificador para mejorar la precisión del sistema propuesto.
Palabras clave: Optimización Metaheurística; Reconocimiento de Emociones (RE); Metaheurística Chiropteran Mahi; E-Walking.
INTRODUCCIÓN
Emotion recognition method recognizes and interprets human emotions based on various behavioral cues, like body language, tone of voice, physiological signals and facial expressions. Emotion recognition has numerous real-world applications, including healthcare, marketing, entertainment, and human-computer interaction. In this document, we will discuss the significance of ER, its applications, and the different techniques used for ER.(1)
Emotions being a critical aspect of decision-making & human behavior, the emotion recognition can help us understand how people feel and respond appropriately.(2)
Techniques Used for Emotion Recognition
Following are the various techniques used for emotion recognition:
• Facial Expression Analysis
• Speech Analysis
• Physiological Signals Analysis
• Multi-Modal Analysis
Figure 1. Input Image: Sad, Angry, Happy
Challenges in Emotion Recognition
There are several challenges(7) in emotion recognition, including the high variability in human emotions, the difficulty in detecting subtle emotional cues, and the impact of context and individual differences. Additionally, the correctness of emotion recognition algorithms can be impacted by various aspects like noise, lighting conditions and the quality of the input data. These challenges highlight the need for robust and accurate emotion recognition algorithms that can handle the complexity and variability of human emotions.
Steps in Emotion Recognition
Emotion recognition covers ascertaining and classifying the emotional state of a person basis his/ her facial expressions, physiological signals, speech and behavior. The following are the steps involved in emotion recognition (as shown in figure 2):(8)
Figure 2. Steps Involved in Emotion Recognition
ER involves collecting data, extracting features, selecting relevant features, training a machine learning design, evaluating the model, and deploying it in a real-world application. Emotion recognition has numerous uses in healthcare, human-robot interaction, entertainment and education industries.
METAHEURISTIC OPTIMIZATION
Metaheuristic optimization is a powerful computational approach for solving complex optimization problems
Metaheuristic optimization approaches are designed to fix complex optimization issues that cannot be solved using traditional methods.(9) It can handle high-dimensional search spaces, where traditional optimization methods become computationally intractable. Moreover, it is often very effective in finding global optima, even in the presence of multiple local optima and finds good solutions quickly, making it a powerful tool for applications where time is a critical factor. Figure 3 shows the hierarchy of metaheuristic optimization algorithms in detail.
Figure 3. Hierarchy of Optimization Algorithms
CHIROPTERAN MAHI METAHEURISTIC OPTIMIZATION (CMMA)
The CMMA algorithm is based on the foraging behavior of bats, where they use echolocation to navigate and find food. In CMMA, the optimization problem is represented as a set of bats, each with a position and velocity in the search space.(10) The bats communicate with each other through echolocation, and the algorithm updates the position of the bats based on their individual fitness and the best fitness found so far.
Dealing with high variability in human emotions is one of the key challenges in emotion recognition. For example, the same facial expression can represent different emotions depending on the context and individual differences. This challenge can be addressed by the CMMA algorithm by optimizing the parameters of the ML designs to capture the subtle variances in the input features that are suggestive of various emotions. Another challenge in emotion recognition is dealing with noisy data, such as data collected in real-world settings.
LITERATURE SURVEY
The most effective methods for emotion recognition are reviewed in this section based on their contribution, advantages, & disadvantages.
Nagpal et al.(11) demonstrated emotion recognition in a noisy setting using a cascade of MBFO and AMF. The method first removes noise from the picture by combining MBFO and AMF & then it recognizes local, global, statistical features in the picture.
Appati et al.(12) used the PC Approach with the intrinsic characteristic of dimensionality reduction for feature selection. To compare the performance, PSO approach, GA & ABC were used for optimizing the resulting characteristics. The optimized characteristics were utilized for identification using Euclidean distance (EUD), KNN & SVM as classifiers.
Yildirim et al.(13) proposed a Feature Selection technique which changes the primary population generation phase of metaheuristic search (MS) approaches. Two MS approaches non dominated sorting genetic algorithm-II (NSGA-II) and Cuckoo Search (NSGA-II & CS) for speech ER using the EMO-DB & IEMOCAP databases for analyzing the technique.
Habibullah et al.(16) reflects a study on facial behavior evaluation to identify depression from facial action units derived from pictures. Authors used a metaheuristic method to identify a smaller set of facial action unit characteristics. The best predictors were chosen using particle swarm optimization and fed into optimized standard feedforward NN.
Oloyede et al.(17) proposed a face recognition system that takes into account all of the limitations in the face dataset. In addition, for effective extraction of unique characteristics from the improved face set of data, a new set of hybrid properties consisting of the Pyramid Histogram Orientation Gradients (PHOG), Edge Histogram Descriptor (EHD), Local Binary Pattern (LBP) has been suggested. Studies on standard face dataset were convey to verify the enhancement in the accuracy of the face identification device that takes into account all of the limitations in the face dataset.
Ashok et al.(18) created an innovative FER scheme utilizing the optimal FS as well as optimal hidden neuron focused NN. The NN was fed with the best-chosen features, Multi-Verse based Whale Optimization Algorithm (MV-WOA) was used for developing hidden neurons of whose. The final findings have demonstrated that FER design suggested has conducted high eras contrasted to conventional methodologies for ER for classifying 7 emotions like "normal, smile, sad, surprise, anger, fear, and disgust".
Abdullah et al.(19) suggested and executed an IR device based on PCA, genetic approaches, NN, in which PCA diminishes the size of the testing set, while evolutionary operators & NN develop image matching searching designs & present high efficiency output in a less period of span.
Prachi et al.(20) suggested Deep BiLSTM system developed utilizing CMO method which detected human feelings successfully. For the proposed emotion recognition based methodology, gaits, feelings or movies serve as an initial input. The most significant features which were extracted and concatenated are fed in the BiLSTM classified for training. Suggested Chiropteran Mahi optimization method provides improved results for recognizing classifier’s revealed emotions by tuning hyper parameters successfully.
PROPOSED CHIROPTERAN MAHI OPTIMIZATION
The proposed Chiropteran Mahi Optimization integrates various echolocation properties used for foraging. The remarkable echolocation abilities of chiropterans enable them to locate prey and identify various flying insects above them, even in the absence of light. This Optimization method based on chiropterans demonstrates superior performance in resolving optimization challenges. One of the key benefits of using this method is that it has a rapid convergence rate during the initial iterations, and it shifts its position from the exploration to the exploitation phase quickly. The switching mechanism used in the chiropteran optimization has some drawbacks on its convergence rate. High loudness and pulse rate of the chiropteran are there during the initial foraging stage which suddenly drop after a few iterations, slowing down the convergence speed. To address this issue, the Mahi Delphinus (MD) utilizes an echolocation technique that generates "clicks" instead of sound, based on pulse rates, during the exploring stage. This helps mitigate the problem. The Mahi Delphinus has a fascinating behavior of exchanging information with its group, which helps to update their location. Vocalizations are primarily made in the form of "clicks" and "whistles." Furthermore, the Mahi Delphinus generates new ideas with varying frequencies to improve their predatory process.
Inspiration
The chiropteran algorithm(14) is a meta-heuristic based algorithm mostly utilized for global optimization due to the outstanding echolocation performance of micro bats. To detect the location and distance of prey or food, the algorithm emits sound with varying pulse rates. The emitted pulse during echolocation does not need to be sustained for a long time; it typically lasts for only a few thousand seconds. The frequency range depends on the wavelength of the emission, and the constant frequency is generally in the range of 25 to 150 kHz, and a wavelength of 2 to 14 mm. Echolocation is the primary physiological method used to estimate the location of prey, undistinguishable objects, or distance by which sound waves are echoed back to the emitter. Chiropteran mammals possess remarkable abilities to distinguish between the appearance of prey or food and obstacles in their path. These abilities make the chiropteran mammal an efficient solution for non-linear related problems, as it is easy to implement and can automatically control emission rate and intensity level to select the auto-zooming region. However, despite these advantages, the convergence rate of the chiropteran algorithm slows down quickly, which makes it unsuitable and results in inadequate accuracy for a smaller number of function evaluations. These problems are resolved by adding MD behavior, like employing fewer components or reducing the randomized searching space.
The MD is capable of producing loudness through clicks, which are generated at a high frequency higher than that used for communication.(15) During the foraging phase, it initially produces sounds that strike a particle, and the resulting sound waves are reflected within the MD. Whenever the transmitter receives the echo, it often produces a second click to accurately estimate the position of any barriers or potential prey. The reflected echo and generated click exhibit time variance, which can be evaluated to determine the residual distance from the object. The signal reflecting off the emitter's head has varying strength, which assists in estimating the direction of travel.
Characteristics of Chiropteran
During the flying mechanism for searching prey, it prefers to fly at random velocity V1 at a specific position PI with the variable wavelength λ and fixed frequency Fmin with the loudness at L0. The pulse generated by the emitted frequency and also the emitted rate of pulse R∈[0, 1] is automatically attuned concentrating on the dist. of the target. Since the loudness is varied in many different ways, thus, the loudness is considered from L0 to the Lmin, where the L0 is the positive range and the constant value is Lmin.
Pulse frequency
The velocity VI and position pI of each bat (I) are described in the D search space and it is consequently renovated for each iteration. The updated remedies at the time step T of the location pIT and velocities VIT are evaluated by the following formulas,
where, δ is the random vector derived from the uniform distribution and it is in the range of [0,1], after the termination of comparing all the best solutions between the l number of chiropterans, the best optimum position is obtained and it is expressed as P*. According to the kind of problem at hand, either as the wavelengths or the rate are changed while holding all other variables constant or raising the velocity. For implementation, the frequency of the sound waves is placed in between the minimum Fmin and maximum Fmax points, which depends on the problem of interest domain size.
Random stage
Initially, the frequency is assigned uniformly in the range of [Fmin, Fmax] for a particular bat. After choosing the best optimal solutions from all available methods in the local foraging zone, the fresh update solution is generated separately for the chiropteran using the random walk method.
PT+1= PT+εLT (4)
The sample proportion is in the between of [-1,1] and it is described as ε, while the following reflects the mean chiropteran intensity at the sampling interval:
LT=<LTI>
The speed and position of chiropteran are renewed, which has some similarities with standard particle swarm optimization. The resemblances are depending on the frequency FI, which regulates the rapidness and range of the moving swarm particles. For controlling the intensity and the rate of the pulse, the chiropteran-based optimization is chosen as the optimal solution based on the well-adjusted fusing structure of local foraging and the standard particle swarm optimization. L is the representation of average loudness.
Varying loudness
If the chiropteran generates the goal, its frequency of pulse emission is increased, eventually reducing the noise levels. Based on the value of fitness, the intensity of the sound is selected, for example, the loudness is assumed as L0=1 and Lmin=0. As long as the loudness minimum value is zero, the chiropteran identifies the prey without generating sound.
where, the constants are represented as β and α, for eg 0<β,α<1:
The benefits of the chiropteran metaheuristic-based optimization in the electronic control stage of the exploitation & exploration include frequency tuning or internal control responsibilities. The amount is selected as for the most straightforward scenario. It is worth noting that even though there are several advantages in the proposed system, however, it is rapidly losing convergence rate, resulting in a decrease in its accuracy rate. In the chiropteran-based algorithm, Mahi Delphinus' excellent foraging behavior can overcome this problem.
Emerging Stage
Integrating the chiropteran's appropriate result with the MD's exploratory activity dramatically boosts CR and precision throughout the early phase. Formula (7) formulates the initial answer, while formula (8) uses the MDGS behavior.
Here S2 signifies the searching space of the MD. Hybridizing equations (7) and (8) for enhancing the performance of the Chiropteran by eliminating the rapid slowdown in the convergence rate, increasing accuracy in the random searching phase, and revealing the optimal solutions.
The pseudocode arranged for the developed chiropteran Mahi metaheuristic-based optimization is explained in algorithm 1.
PRELIMINARY SETUP
PYTHON is implemented in Windows 10 operating system having 8 GB of RAM for demonstrating the efficiency of the suggested Chiropteran Mahi Metaheuristic method of optimization.
Dataset
An "EWalk (Emotion Walk)" dataset is presented in which the observed emotions are identified from walking videos of individuals using the labeled and gait emotions present in the video.
Performance Evaluation
The various evaluation criteria needed to use the created approach to detect people's emotions while they're moving are as follows:
Accuracy: the percentage of real positive & negative responses across all tests can reliably predict a person's emotional state.
Acc=True(positive+negative) / Total Samples (10)
Sensitivity: the proportion of correct forecasts to the total of accurate or misclassified forecasts determines the suggested technique's sensitivity.
Sen = Precise predictions / Total number of precise and misclassified predictions (11)
Specificity: the proportion of correct true negative forecasts to all true negative or failed predictions is used to determine the specificity of the proposed methodology.
Speci = (Accuarte prediction of TN) / (Complete set of true negative & False Positive Predictions) (12)
CONCLUSION AND FUTURE SCOPE
In conclusion, ER is a crucial area in the region of human-computer interaction, with various actual-world apps. Emotion recognition helps us in recognizing and inferring human emotions on the basis of several behavioral indications, like physiological signals, facial expressions & voice tone. Physiological signals analysis, facial expression analysis, multi-modal analysis and speech analysis are some of the techniques utilized for ER. Despite the challenges in emotion recognition, ER has the capability to transform various domains, like healthcare, education, marketing & entertainment. This paper intrudes Chiropteran Mahi Metaheuristic Optimization, which is a promising optimization algorithm for emotion recognition, as it can optimize the parameters of machine learning models and help in enhancing the ER system’s robustness & accuracy. The CMM algorithm is likely to become a valuable tool for emotion recognition research and development as the human-computer operation field continues to grow.
BIBLIOGRAPHIC REFERENCES
1. Leh Luoh,; Chih-Chang Huang, ; Hsueh-Yen Liu, “Image processing based emotion recognition,” International Conference on System Science and Engineering, pp. 491–494, 2010. doi:10.1109/icsse.2010.5551816, 2010.
2. Puri, R., Gupta, A., & Saikri, M., “Emotion Detection using Image Processing in Python,” 5th International Conference on “Computing for Sustainable Global Development, pp. 1389-1394,2018.
3. Prakash, U. M., Pratyush, Dixit, P., & Ojha, A. K., “Emotional Analysis Using Image Processing,” International Journal of Recent Technology and Engineering (IJRTE), vol. 7, no. 5S2, pp. 258-262, 2019.
4. Bhadane, A., Dixit, A., Ingle, V., & Shastri, D., “Facial Expression Recognition Using Image Processing, IJARIIE, vol. 5, no. 3, pp. 266-271, 2019.
5. Khalil, Ruhul Amin; Jones, Edward; Babar, Mohammad Inayatullah; Jan, Tariqullah; Zafar, Mohammad Haseeb; Alhussain, Thamer, “Speech Emotion Recognition using Deep Learning Techniques: A Review,” IEEE Access, vol. 7, pp. 117327 - 117345, 2019. doi:10.1109/ACCESS.2019.2936124, 2019.
6. Saxena, A., Khanna, A., & Gupta, D., “Emotion Recognition and Detection Methods: A Comprehensive Survey,” Journal of Artificial Intelligence and Systems, vol. 2, pp. 53–79, 2020. https://doi.org/10.33969/AIS.2020.21005.
7. Mohanta, S. R., & Veer, K., “Trends and challenges of image analysis in facial emotion recognition: a review,” Network Modeling Analysis in Health Informatics and Bioinformatics, vol. 11, 2022.
8. Kołakowska, A., Landowska, A., Szwoch, M., &Szwoch, W., “Emotion recognition and its application in software engineering,” 6th International Conference on Human System Interaction, 2013.
9. Passos, L. A., & Papa, J. P., “ A metaheuristic-driven approach to fine-tune Deep Boltzmann Machines,”Applied Soft Computing, 2019.
10. Altan, A., “Performance of Metaheuristic Optimization Algorithms based on Swarm Intelligence in Attitude and Altitude Control of Unmanned Aerial Vehicle for Path Following,”4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT),2020.
11. Nagpal, R., Nagpal, P., & Kaur, S., “Hybrid Technique for Human Face Emotion Detection,” International Journal of Advanced Computer Science and Applications(IJACSA), vol. 1, no. 6, 2010.
12. Appati, J. K., Abu, H., Owusu, E., &Dakwah, K., “Analysis and Implementation of Optimization Techniques for Facial Recognition,” Applied Computational Intelligence and Soft Computing, vol. 2021, 2021.
13. Yildirim, Serdar; Kaya, Yasin; Kılıç, Fatih, “A modified feature selection method based on metaheuristic algorithms for speech emotion recognition,” Applied Acoustics, vol. 173, 2021. doi:10.1016/j.apacoust.2020.107721
14. Amir Hossein Gandomi, Xin-She Yang, Amir Hossein Alavi, Siamak Talatahari, “Bat algorithm for constrained optimization tasks,” Neural Computing and Applications, vol.22, no.6, pp.1239-1255, 2013.
15. Kaveh, A., and Farhoudi, N., “A new optimization method: Dolphin echolocation,” Advances in Engineering Software, vol.59, pp.53-70, 2013.
16. Habibullah Akbar, Sintia Dewi, Yuli Azmi Rozali, Lita Patricia Lunanta,Nizirwan Anwar, “Exploiting Facial Action Unit in Video for Recognizing Depression using Metaheuristic and Neural Networks,”1st International Conference on Computer Science and Artificial Intelligence (ICCSAI) ,2021.
17. Oloyede, M. O., Hancke, G. P., & Myburgh, H. C., “Improving Face Recognition Systems using a new Image Enhancement Technique, Hybrid Features and the Convolutional Neural Network,” IEEE Access, 2018.
18. Ashok Kumar, P. M., Maddala, J. B., & Martin Sagayam, K., “Enhanced Facial Emotion Recognition by Optimal Descriptor Selection with Neural Network,”IETE Journal of Research, 2021.
19. Abdullah Ayub Khan, Aftab Ahmed Shaikh, Zaffar Ahmed Shaikh, Asif Ali Laghari, “IPM-Model: AI and metaheuristic-enabled face recognition using image partial matching for multimedia forensics investigation with genetic algorithm,” Springer,2022.
20. Prachi Jain, Vinod Maan, “Emotion Recognition Using Human GAIT with Chiropteran Mahi Metaheuristic Algorithm”, Eur. Chem. Bull., 12 (Issue 8), 4671-4685, 2023.
FINANCING
We did not receive financing for the development of this research.
CONFLICT OF INTEREST
We declare that there is no conflict of interest.
AUTHORSHIP CONTRIBUTION
Conceptualization: Vinod Maan.
Data curation: Vinod Maan.
Formal analysis: Prachi Jain, Vinod Maan.
Acquisition of funds: Prachi Jain, Vinod Maan.
Research: Prachi Jain, Vinod Maan.
Methodology: Prachi Jain.
Project management: Prachi Jain, Vinod Maan.
Resources: Prachi Jain, Vinod Maan.
Software: Prachi Jain.
Supervision: Vinod Maan.
Validation: Prachi Jain, Vinod Maan.
Display: Prachi Jain.
Drafting - original draft: Prachi Jain.
Writing - proofreading and editing: Prachi Jain.