5
$\begingroup$

Is anyone aware of any open source / python packages for Netflow Anomaly detection ? I found some on github but anyone who has more experience with it. please advise.

$\endgroup$
0
$\begingroup$

I could not find any specific package in the internet that does the same thing as netflow anomaly detection, but I found out how to train some standard algorithms that solve the same problem. My understanding is that the problem is "network intrusion detection" or "anomaly detection in network traffic to detect network intrusion"

So creation of a model can be divided into several parts:

  1. choosing an appropriate algorithm
  2. understand how to transform the data for the algorithm
  3. finding the data set
  4. training the model

The above tasks in more detail:

  1. algorithm: k-NN or Random Forest. According to the paper Machine Learning DDoS Detection for Consumer Internet of Things Devices k-nearest neighbor is a pretty precise algorithm in network anomaly detection. Nearest neighbor algorithms are present in scikit-learn python package (link). Random forest classifier performed even better. scikit-learn also has a random forest classifier (link).

Here is the precision table from the first paper. KN is k-NN. RF is Random Forest. NN is Neural Network. LSVM is a variation of Support Vector Machine algorithm. Algorithm precision

  1. How k-NN for DDoS works is explained in this paper: Proactive Detection of DDoS Attacks Utilizing k-NN Classifier in an Anti-DDos Framework.

  2. According to the above mentioned papers and other sources the following datasets are used for training:

    1. KDCup1999 This is a network intrusion database
    2. MIT Lincoln Lab, 2000, DARPA intrusion detection scenario specific datasets Contains two DoS attack scenarios
    3. Reasearchgate question on available datasets
  3. On this link you will find how to train a k-nearest neighbor in scikit-learn.

Here is an example code from the link:

>>> X = [[0], [1], [2], [3]]
>>> y = [0, 0, 1, 1]
>>> from sklearn.neighbors import KNeighborsClassifier
>>> neigh = KNeighborsClassifier(n_neighbors=3)
>>> neigh.fit(X, y) 
KNeighborsClassifier(...)
>>> print(neigh.predict([[1.1]]))
[0]
>>> print(neigh.predict_proba([[0.9]]))
[[0.66666667 0.33333333]]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.