Code for anomaly detection and clustering of energy time series (Swisscom/Migros data)

This wonderful repo contains multiple studies done within in the framework of the Anomaly detection project in coordination with Swisscom and OPIT. These studies are essentially represented by a few (python) jupyter notebooks, in which all analyses were conducted. Note that we had access to Swisscom and Migros datasets, which had very different patterns and therefore were treated in separated studies.

Contents of the notebooks:

Notebooks on Swisscom data:

1. "Swisscom_test": notebook with first test with Swisscom data (viz, creation of monthly images, circle representation code, first anomaly detection try out with isolation forests)
2. "Swisscom_withLabels_processingToImages" notebook: First analysis of labeled data (viz, creation and storage of images for later training with CNNS)
3. "Swisscom_withLabels_Training" notebook: training of CNNs with previously extracted images data
4. "Swisscom_data_Fourier_PCA_try" notebook: applying Students methodology for anomaly detection to Swisscom data. Didn’t work.
5. "Swisscom_CLUSTERING" notebook: last tests and results for the swisscom clustering (Fourier features clustering with DBSCAN, 2D simple clustering with moving video, extraction of swisscom internal classes)

Notebooks on Migros data:

1. "Migros_data_tests" notebook: first analysis on migros data, plot images to see how stable the data is…
2. "Migros_CLUSTERING" notebook: clustering tests on the migros data (Fourier features clustering with DBSCAN)

Note that these notebooks call different sources of data, which are all stored in the deneb2 cluster, in /work/hyenergy/raw/Swisscom/dataset_Dan_june2020

Note also that "TrainingMethods188.py" is a python script containing classes and functions to ease the training of SVM and RFs.