Emotion Detection from Electroencephalography Data with Machine Learning : Classification of emotions elicited by auditory stimuli from music on self-collected data sets

Sammanfattning: The recent advances in deep learning have made it state-of-the-art for many different tasks, making its potential usefulness for analyzing electroencephalography (EEG) data appealing. This study aims at automatic feature extraction and classification of likeability, valence, and arousal elicited by auditory stimuli from music by training deep neural networks (DNNs) on  minimally pre-processed multivariate EEG time series. Two data sets were collected, the first containing 840 samples from 21 subjects, the second containing 400 samples from a single subject. Each sample consists of a 30 second EEG stream which was recorded during music playback. Each subject in the multiple subject data set was played 40 different songs from 8 categories, after which they were asked to self-label their opinion of the song and the emotional response it elicited. Different pre- processing and data augmentation methods were tested on the data before it was fed to the DNNs. Three different network architectures were implemented and tested, including a one-dimensional translation of ResNet18, InceptionTime, and a novel architecture built upon from InceptionTime, dubbed EEGNet. The classification tasks were posed both as a binary and a three-class classification problem. The results from the DNNs were compared to three different methods of handcrafted feature extraction. The handcrafted features were used to train LightGBM models, which were used as a baseline. The experiments showed that the DNNs struggled to extract relevant features to discriminate between the different targets, as the results were close to random guessing. The experiments with the baseline models showed generalizability indications in the data, as all 36 experiments performed better than random guessing. The best results were a classification accuracy of 64 % and an AUC of 0.638 for valence on the multiple subject data set. The background study discovered many flaws and unclarities in the published work on the topic. Therefore, future work should not rely too much on these papers and explore other network architectures that can extract the relevant features to classify likeability and emotion from EEG data.