I'm trying out different anomaly detection models and would love to hear opinion on my idea from somebody experienced. My goal is to perform anomaly detection with different models and to give each point in data anomaly score based on the predictions of different models.
A lot of tutorials says that k-means is okay to use for anomaly detection even though it classifies anomalies in clusters. But my thought was to use counter to check the 2-3 clusters that have the least points and to assume that they are anomalous. Could this work out is or is this assumption way too vague?
Another problem is that my data is not labeled so there is no ground truth for me to evaluate the performance of the models. How can I do that or where do I even start?