I'm researching anomaly detection, which is nothing else than outliers detection on a set of time-series web servers access log data or network traffic. Recently I re-faced to following fundamental questions and literature review in this regard to updating myself and find clear answers:
- Deep Learning (DL) is supervised or unsupervised?
- Does splitting data make sense in unsupervised settings?
- How can we calculate the loss function (MSE, MAE, etc.) for the unsupervised setting? Is it common?
Personally, I'm targeting Clustering methods, and its validation is based on similarities as well as isolation forest and move on Autoencoders for DL as recommended here. Of course, I'm aware that Cluster analysis is not something to automate fully, and It is an explorative method. On the other hand, not completely reliable I was always imagining the main reason for distinguishing between supervised and unsupervised is the existence of label features in data. My picture about DL is mostly the high number of hidden layers and leave the responsibility of feature engineering for DL models. I do like this vision concerning comparing unsupervised anomaly models. I believe that we need still to splitting data and calculate loss function despite unlabeled data due to the fact that:
- calculating the metrics on the training set would likely lead to overfitting, and then we need the testing set to evaluate the model. Generally speaking, to track the level of overfitting while we are experimenting with our network.
- get an estimate of how many epochs we should train for
Additionally, I read this online article Issues with Unsupervised Learning and Why still we need them.
I know the above-mentioned questions look naive, but any help or clarification, even in the short form for this trilogy questions, will be highly appreciated and help me get rid of this confusion and shape my vision correctly.