Deep Learning of Affective Content from Audio for Computing Movie Similarities

Sammanfattning: Recommendation systems are in wide use by many different services on theinternet today. Most commonly, recommendation systems use a technique calledcollaborative filtering, which makes recommendations for a user using other users’ratings. As a result, collaborative filtering is limited by the number of user ratingsin the system. Content-based recommendations perform direct comparisons on theitems to be recommended and thus avoid dependence on user input. In order toimplement a content-based recommendation engine, pairwise similarity measuresmust be calculated for all of the entities in the system. When the entities to be rec-ommended are movies, it can be informative to make comparisons using emotional(or affective) content. This work details the investigation of different methodologiesfor extracting affective content from movie audio using deep neural networks. First,different types of feature vectors in concert with a variety of model parameters fortraining were examined in order to project input audio data into a three dimen-sional valence-arousal-dominance (VAD) space where affective content can be moreeasily compared and visualized. Finally, two different similarity measures for directcomparison of movies with respect to their affective content were introduced.