A Multivariate Data Stream Anomaly Detection Framework

Sammanfattning: High speed stream anomaly detection is an important technology used in many industry applications such as monitoring system health, detecting financial fraud, monitoring customer's unusual behavior and so on. In those scenarios multivariate data arrives in high speed, and needs to be calculated in real-time. Since solutions for high speed multivariate stream anomaly detection are still under development, the objective of this thesis is introducing a framework for testing different anomaly detection algorithms.Multivariate anomaly detection, usually includes two major steps: point anomaly detection and stream anomaly detection. Point anomaly detection is used to transfer multivariate feature data into anomaly score according to the recent stream of data. The stream anomaly detectors are used to detect stream anomalies based on the recent anomaly scores generated from previous point anomaly detector. This thesis presents a flexible framework that allows the easy integration and evaluation of different  data sources, point and stream anomaly detection algorithms. To demonstrate the capabilities of the framework,  we consider different scenarios with generators of artificial data, real industry data sets and time series data, point anomaly detectors of PYISC, SVM and LOF, stream anomaly detectors of DDM, CUSUM and FCWM. The evaluation results show that for point anomaly detectors, PYISC and LOF perform well when the distributions of features are known, SVM performs well even when the distributions of features are not known. For the stream anomaly detectors, DDM has some possibilities to get false anomaly detection, CUSUM has some possibilities to get failed when the stream anomalies increase slowly, while FCWM performs best with very low possibilities to get failed.