Anomaly Detection in ConsoleLogs

Sammanfattning: The overall purpose of this project was to find anomalies inunstructured console logs. Logs were generated from system componentsin a contact center, specifically components in an email chain. Ananomaly is behaviour that can be described as abnormal. Suchbehaviour was found by creating features of the data that later oncould be analyzed by a data mining model. The mining model involvedthe usage of normalisation methods together with different distancefunctions. The algorithms that were used in order to generate resultson the prepared data were DBSCAN, Local Outlier Factor, and k-NNGlobal Anomaly Score. Every algorithm was combined with two differentnormalisation technologies, namely Min-Max- and Z-transformationnormalisation. The six different experiments yielded three datapoints that could be considered anomalies. Further inspection on thedata showed that the anomalies could be divided into two differenttypes of anomalies; system- or user behavioural related. Two out ofthree algorithms gave an anomaly score to a data point, whereas thethird gave a binary anomaly value to a data point. All the sixexperiments in this project had a common denominator; two data pointscould be classified as anomalies in all the six experiments.