Intelligent System for the Classification of Mental State Parameters

Mental health is essential for overall well-being, focusing emotional, psychological, and social aspects. Assessing and managing mental health requires understanding mental state parameters, including cognitive load, cognitive impairment, and emotional state. Advanced technologies like eye tracking provide valuable insights into these parameters, transformed mental health evaluation and enabled more targeted interventions and better outcomes. Thesis focused towards developing intelligent system to monitor mental health, focusing on cognitive load, cognitive impairment, and emotional state. The research has three main objectives, including creating four eye-tracking-based unimodal datasets and a multimodal dataset to address the lack of publicly available mental health assessment datasets. Each dataset is designed to study cognitive load, cognitive impairment, and emotional state classification using varied stimuli. In addition to dataset creation, the thesis excels in feature extraction, introducing novel features to detect mental state parameters and enhancing assessment precision. High-level features such as error rate, scanpath comparison score, and inattentional blindness are incorporated, contributing to find cognitive impairment scores. Five models are developed to detect mental states by separately monitoring the mental state parameters, cognitive load, cognitive impairment, and emotional state. The models employ statistical analysis, machine learning algorithms, fuzzy inference systems, and deep learning techniques to provide detailed insights into an individual's mental state. The first two models, Eye-Tracking Cognitive Load models (ECL-1 and ECL-2) focus on cognitive load assessment during mathematical assessments and Trail Making Test tasks. ECL-1 model utilizes statistical analysis to understand the correlation between eye tracking features like pupil diameter and blink frequency with the cognitive load while performing mathematical assessments. With the identification of relevant features while performing Trail Making Test (TMT), the ECL-2 model effectively classifies low and high cognitive load states with a notable 94% accuracy, utilizing eye-tracking data and machine learning algorithms. The third model, the ETMT (Eye tracking based Trail Making Test) model, uses a fuzzy inference system and adaptive neuro-fuzzy inference system to detect mental states associated with cognitive impairment. It provides detailed scores in visual search speed and focused attention, important for understanding the exact cognitive deficits of a patient. This greatly aids in understanding the cognitive states of an individual and addresses deficits in executive functioning, memory, motor function, attentional disengagement, neuropsychological function, processing speed, and visual attention. The fourth model, PredictEYE, utilizes a deep learning time-series univariate regression model based on Long Short-Term Memory (LSTM) to predict future sequences of each feature. Machine learning-based Random Forest algorithm is applied on the predicted features for mental state prediction and identifying the mental state as calm or stressful based on a person's emotional state. The personalized time series methodology makes use of the power of time series analysis, identifying patterns and changes in data over time to enable more precise and individualized mental health assessments and monitoring. Notably, PredictEYE outperforms ARIMA with an accuracy of 86.4%. The fifth model introduced in this study is based on a multimodal dataset, incorporating physiological measures such as ECG, GSR, PPG, and respiratory signals, along with eye tracking data. Two separate models, one based on eye tracking data and the other based on all other physiological measures developed for understanding the emotional state of a person. These models demonstrate comparable performance, with notable proficiency in binary classification based on arousal and valence. Particularly, the Binary-Valence model achieves slightly higher accuracy when utilizing eye tracking data, while other physiological measures exhibit stronger classification performance for the Binary-Arousal model. The thesis makes substantial progress in mental health monitoring by providing accurate, non-intrusive evaluations of an individual's mental state. It emphasizes mental state parameters such as cognitive load, impairment, and emotional state, with AI-based methods incorporated to improve the precision in detection of mental state.