I am new to machine learning and have finished Andrew Ng's course on Coursera. I have just begun to tackle my first "real" ML problem - which is a binary classification problem. I was wondering if the steps I have taken is the "usual" way to work through a problem.
I have gathered my data which is in this case is some images (positive and negative images)
Preprocess the data - in my case read the images and convert these to grayscale images in order to simplify the data. I have then set up my arrays in terms of input and labels. No feature scaling is needed in my opinion or anyother preprocessing.
After this I split the data in training and test set (80-20 ratio). I used stratified split.
Model selection and evaluation. In this case I used three models: logistic reg., SGDClassifier and a decision tree.
Train the different models and evaluate them using k-fold cross validation sets. I evaluated the mean accuracy on the CV set and also the F1-score on the CV set. Based on this I selected the best modell (the scores for all the modells on the training set was 1 and on the CV sets 0.91 (log.reg) , 0,57 (dec.tree) and 0.73 (SGD))
NOTE : Shouldn't I plot some learning curves too see if there is a overfitt or underfitt problem (based on the result from the CV sets it seems like log.reg model is a good fit). Is there anything else to study or is it enough based on the scores?
Fine tune the selected model.
Evaluate on the test set.