0
$\begingroup$

In the code I upon a csv file which only has one column. The data in there in not that important just normal numbers.

# Pandas - data handling
import pandas as pd

# Numpy for mathematical operations
import numpy as np
import pandas as pd

# Scikit learn for the DBSCAN algorithm
from sklearn.cluster import KMeans

# Matplotlib - plots
import matplotlib.pyplot as plt

import seaborn as sns


data=pd.read_csv('C:/Users/data.csv', usecols=["meaning_txt", "valueData_value"], sep=';' )

# Add manually an anomaly value
for i in range(0, 50):
    all_data = all_data.append({all_data.column.max() + np.random.randint(1000, 2000) }, ignore_index=True)

clf = KMeans(n_clusters=2, init='k-means++', max_iter=300, n_init=10, random_state=1)

clf.fit(all_data.column.values.reshape(-1, 1))

all_data_prd = clf.predict(all_data.column.values.reshape(-1, 1))
all_data_prd

plt.scatter(all_data_prd, all_data.valueData_value, c=all_data_prd)

My question is if it is possible to have a confusion matrix and roc curve in this case when I only have once column.

To this column I have added some anomalies.

The goal:

  1. Add these anomalies that are way way bigger than the maximum value of the column and then check how many out of the 50 added anomalies, how many the K-Means finds
  2. Add these anomalies that are not that much bigger than the maximum value of the column then check how many out of the 50 added anomalies are found. At the end I want a Confusion matric and a ROC graph to see how good they perform.
$\endgroup$
  • 1
    $\begingroup$ It looks to me like there are lots of problems with this approach: (1) I doubt k-means is a good approach for one-dimension data, there are certainly better ways to detect anomalies (2) confusion matrix and ROC are evaluation measures for classification, so it's not going to work directly with the output of k-means. (3) since you know what is regular data and anomalies, why not using supervised classification? (4) Finally if the goal is just to find the threshold which is optimal to separate regular cases vs anomalies, you might not even need ML. $\endgroup$ – Erwan Mar 6 at 13:05
  • $\begingroup$ @Erwan, thank you for the very detailed comment. I had some questions would appreciate if you could give me a feedback, if not still thank you. 1. What other methods/algorithms or approaches would you suggest for this case ? 3. What difference would supervised classification make in this case and which exact method you had in mind ? 4.what could I use then ? $\endgroup$ – be1995 Mar 6 at 19:39
2
$\begingroup$

It's difficult to answer precisely without knowing the data and the task.

Assuming it's a single column of values with no order involved, it boils down to finding the optimal threshold to separate regular values vs. anomalies. Given that you know which ones are the anomalies (labelled data), the problem can be treated as a binary classification task: a value is either regular (negative) or an anomaly (positive).

The idea is simply to evaluate the performance for every possible threshold, i.e. count the number of True/False Positive/Negative for every value. Let's say you take the set $S$ of all the distinct values in your column. For every value $t \in S$ you calculate the confusion matrix (i.e. how many TP/FP/TN/FN) obtained by predicting every value $x$ as positive if if $x>t$, negative otherwise.

From there you can plot a ROC curve and/or find the optimal threshold according to some evaluation measure (typically the one which maximizes the F1-score).

I assume that there are libraries which do that automatically, but it's simple enough so personally I would do my own code for this kind of thing.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.