0
$\begingroup$

I'm researching anomaly detection, which is nothing else than outliers detection on a set of time-series web servers access log data or network traffic. Recently I re-faced to following fundamental questions and literature review in this regard to updating myself and find clear answers:

  • Deep Learning (DL) is supervised or unsupervised?
  • Does splitting data make sense in unsupervised settings?
  • How can we calculate the loss function (MSE, MAE, etc.) for the unsupervised setting? Is it common?

using python & sklearn:

from sklearn.metrics import mean_squared_error
#Feature Selection
criterion = mean_squared_error(y, predictions)

or numpy:

import numpy as np
criterion = np.mean((y_test - est.predict(X_test))**2)

Personally, I'm targeting Clustering methods, and its validation is based on similarities as well as isolation forest and move on Autoencoders for DL as recommended here. Of course, I'm aware that Cluster analysis is not something to automate fully, and It is an explorative method. On the other hand, not completely reliable I was always imagining the main reason for distinguishing between supervised and unsupervised is the existence of label features in data. My picture about DL is mostly the high number of hidden layers and leave the responsibility of feature engineering for DL models. I do like this vision concerning comparing unsupervised anomaly models. I believe that we need still to splitting data and calculate loss function despite unlabeled data due to the fact that:

  1. calculating the metrics on the training set would likely lead to overfitting, and then we need the testing set to evaluate the model. Generally speaking, to track the level of overfitting while we are experimenting with our network.
  2. get an estimate of how many epochs we should train for

Additionally, I read this online article Issues with Unsupervised Learning and Why still we need them.

I know the above-mentioned questions look naive, but any help or clarification, even in the short form for this trilogy questions, will be highly appreciated and help me get rid of this confusion and shape my vision correctly.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ If I understand correctly, you have no true labels of what is or is not anomaly? In that case, there is no concept of loss or error, since you have absolutely no way of telling whether your output is correct or not - there's nothing to compare your predictions against. Any loss/error function requires you to know how far off the prediction is from the "correct" answer, but you can't do that without knowing the correct answer. $\endgroup$
    – Nuclear Hoagie
    Commented Dec 3, 2020 at 15:01
  • $\begingroup$ Right, Thanks for your quick input. Can we say that DL or Clustering methods do this labeling job internally with some amount of error we try to detect anomalous or abnormalities? so What are your short answers to bullet Qs generally aside anomaly concept? What is the state-of-the-art then? I know one approach could be to use PU learning or label some part of data anyway as it is mentioned in this post $\endgroup$
    – Mario
    Commented Dec 3, 2020 at 15:20

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.