I am working with a corpus that has 5 datasets in product reviews (A, B, C, D and E), mine is a text classification problem and I need to find the best 5 top models in terms of classification performance (F1).
I started with collection A: the mp3 reviews, Because it has the largest numbers of documents (900: yes, 750: No).
I trained the data using 10-fCv using different algorithms and pre-processing tasks, got the weighted results for all experiments.
I chose the top 5 models and I want to apply them to the rest of the corpus: B, C, D and E (other products' reviews).
My plan is to run 10-fCv and get the results for all the collections and compute the Micro-average for precision, recall and F1.
Is this the right way to choose a model for a large collection?