1
$\begingroup$

I want to detect anomalies in the bank data set in an unsupervised learning method. However, in the bank data set, all columns except time and amount were categorical data, and about half of them had more than 90 percent missing values.

This data set tries to detect anomalies through unsupervised learning. I'm currently using Autoencoder to access it, but I wondered if this would work. Also, because the purpose is to detect whether data is abnormal when data comes in in real time, clustering techniques such as dbscan are limited.

If you want to apply this categorical and many missing values to unsupervised learning, I am curious about how to organize the data and whether there is an appropriate unsupervised learning method.

Since the dataset is the data used in the project, we don't currently have it.

$\endgroup$
1
  • $\begingroup$ An IsolationForest approach might be better for categorical data, although I'm not sure about the performance when it comes to a lot of missing values. $\endgroup$
    – N. Kiefer
    Commented May 17, 2021 at 8:43

1 Answer 1

1
$\begingroup$

One option is the Isolation Forest algorithm. Since it is tree-based, it can handle categorical features.

The rate of missing data will pose a major problem to the interpretation of any model. It might be difficult to tell if the data missing at random or if the missing data is related to the anomalies.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.