0
$\begingroup$

I have a dataset split into training, validation, and testing sets. I trained this model on the training data and evaluated on the validation and testing sets. Now I have an additional set of data that was collected from a separate source on which I have also assessed the classifier. I'm interested in testing whether the gain in performance compared to a random model is similar between the two datasets which have different numbers of positive samples. Is there a procedure that is standard to employ in such a case? Anything more robust than simply comparing {classifier test set #1 performance - random performance} to {classifier test set #2 performance - random performance}?

Improve this question
$\endgroup$

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.