Skip to main content
rob_med's user avatar
rob_med's user avatar
rob_med's user avatar
rob_med
  • Member for 7 years, 10 months
  • Last seen more than 1 year ago
awarded
awarded
awarded
revised
Loading…
awarded
awarded
awarded
awarded
answered
Loading…
awarded
revised
Methods to detect this kind of outliers
Added anomaly-detection tag.
Loading…
suggested
Approve
Loading…
answered
Loading…
answered
Loading…
comment
My network optimises for mean_squared_error, but the predictions are useless
So the network is predicting the most frequent classes, that is 6-7, given the skewness. It also looks like your data is ~6000 samples, and you are using 10000 features. If this is the case, you're most likely also overfitting on the training data, since it is too wide (the model has way to many parameters w.r.t. the number of samples). You can try either reducing the number of lemmas you use, or gathering more data (but you'd need quite some more). Personally, I would start with a small set (say 50 reviews x rating) and ~100 lemmas and see how that goes.
comment
My network optimises for mean_squared_error, but the predictions are useless
What is the frequency of labels in your data? If this is really skewed this might be the problem. Another think you can try is to make the problem easier, i.e. try to predict 4 classes -- for example, rating 1-3, 4-5, 6-7, 8-10 (very bad, bad, ok, good) and see if this gives good result. In that case, you just might need more data for extending to the 10 classes.
revised
Loading…