My method of evaluating a model is the following :
- Split the training data set and do cross validation to obtain an accuracy of my model on my cross validation data set.
- Use the parameters that gave me the best accuracy and use predict() on my test data set ( hold-out data set )
- Run a little 'for' loop to check how many labels i missclassified ( Let's assume i'm doing classification ) on my test dataset for which i hid the real labels.
- I'd look at the percentages of 'accuracy' given by each algorithm and pick the best accuracy.
My question :
What can be done to improve my method of error-analysis and model evaluation using Python? Code snippets and their purposes would be helpful.