I am playing with a dataset that contains tripadvisor restaurant reviews and their labels (either 1, 2, 3, 4 or 5 stars).
Initially I was thinking of using it as a classification problem, applying softmax, cross-entropy and so on, however upon a second thought it came to me the idea to actually treat it as a regression problem, as generating an output which is continuous (such as 3.897) is perfectly fine (assuming of course that the original value would have been 4).
Does this make sense? or should I just hot-encode the answers in a vector of 5 elements (on per star?).
Also, assuming that regression is a correct way to treat this, linear activation in the last layer would be the way to go, and the metrics to look at would be the mean squared error and the mean absolute error.
After browsing a bit more, I found something named Ordinal Regression, actually if one thinks a bit about it, everything makes sense. If i just treat the problem as a classification one, and I have a prediction saying that a given review has 1 star when in reality it was 4 star, how would we penalize that?
- If we are using a classification problem and we are using cross entropy, that is simply a missclassification error, and labeling something as 1 start when it should have been a 4 start is consider as incorrect as labeling something with 3 stars when it should have had 4 stars. As you can see this is incorrect, both predictions (1 and 3 stars) are incorrect, but one is waaay more incorrect than others.
- However if we are using a regression problem, our cost function will clearly penalize more a prediction of 1 start rather than one of 3 stars (again, here we assume that the correct label would have been 4 stars).
So given all that, I would say that regression makes sense in this case, as the categories are related to each other.