I am looking into creating a model to predict whether an item is "Very Good", "Good", "Bad" or "Very Bad".
After I fit the training data to the models, comparing the accuracy of the models during test stump me: should it matter if a model misclassified a G to VG while the other G to VB? What about a model that has two misclassifications of one level away versus another model with only one misclassification but three levels away (eg VG to VB)?
Any guideline on what is the common approach? Also, my thinking at the moment is that this should be a regression problem, but I'm happy to be corrected if I should approach this labeling of datasets more as a classification problem.