I have a multi-class classification problem to solve which is highly imbalanced. Obviously I'm doing oversampling, but I'm doing cross-validation with the over-sampled dataset, as a result of which I should be having repetition of data in the train as well as validation set. I'm using lightgbm algorithm, but surprisingly there is not much difference between cross-validation score and the score on the unseen dataset.
However I just want to know whether its fine to do cross-validation after oversampling the dataset, if not why am I getting such close score on the validation set and the unseen test set?
Also if its not correct to do oversampling before the cross-validation, then it becomes to lengthy to split the data into validation and training and then again sample the training set, and again during final prediction if you're looking to use all the data then you've to append the validation and the training data and then again oversample. Is there any shortcut method to solve the problem?