0
$\begingroup$

I have a sample of data I'd like to create a model from, which would create N clusters. After the fitting to clusters, I'd like to test various samples against the existing clusters, seeing if the samples fit any of the clusters, and if not see it as an "anomaly". I started working on it using DBScan (scikit), my data splits into 3 clusters but I can't find any function/method to test the specific sample against current results, without "refitting". Is there any other model that would fit my needs? or am I missing something when using DBScan?

Thanks in advance

$\endgroup$
1
$\begingroup$

The way to do anomaly detection with clustering is to compute the distance to each the fitted clusters. If a new sample has a distance above a certain threshold for all clusters, then it is considered an anomaly.

For KMeans the distance would be the distance to the center of the cluster. And the threshold to use is a new hyperparameter.

For DBSCAN one would typically compute the distance to each Core point in a cluster, and find the smallest distance. DBSCAN already has a distance threshold eps used during fitting. But to allow tuning sensitivity of the anomaly detection without redoing the clustering, you may want to use a threshold of eps*ratio, where ratio is a new hyperparamter.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.