I am working on a project where I train anomaly detection algorithms Isolation Forest and Auto-Encoder. My data is labelled so I have the ground truth but the nature of the problem requires unsupervised/semi-supervised anomaly detection approach rather than simple classification. Thus I will use the labels for validation only.
Since I will not train the model with the labels, unlike supervised learning where I would have X_train, X_test, y_train and y_test, what is the right approach for model validation here?
If this were supervised learning, I would split data into 3 parts: train, CV and test, doing K-Fold CV. But now I feel like I can simply divide my data into 2: train and test and simply fit all of the train data, predict and tune the models according to. Then finally, predict the test data.
So my question is, should I include some kind of CV in this model? What is the right way here?