New answers tagged

0

You should frame your problem as ordinal regression. Then the model would predict target value, one of the five integer values. As a result, the evaluation would not be best done with Root Mean Square Error (RSME). Chi-Square test could be applied between expected and predict counts for each of the five value levels. If you want to then add in other model ...


0

There's no standard range of values because evaluation scores are never good or bad in absolute, they are relevant with respect to a reference. The standard way to report evaluation scores in a paper is to present them in the context of other methods for the same task: If there are other results about the same task (or a similar task) in the literature, ...


0

Firstly, lets suppose model omits the 'Size' as most significant feature, so what is implied here, having larger size or lower size of an app contribute to the rating? What If there is no ascending or descending order in the attribute, for instance, if the 'Category' is most significant, then what category contributed the most? Decision Tree splits the ...


0

Precision, recall and F1 score are defined only for the binary case (2 classes), so if you want to apply that to the multiclass case, you need to apply a trick. A typical trick is to average the recall per class: Per class, you calculate which fraction of the words actually in that class are correctly classified. balanced_accuracy_score() in scikit-learn ...


0

I have dealt with the same problem. No, you cannot use generated samples, especially with an algorithm such as SMOTE, as it can be bad for accuracy and precision. SMOTE does not take into account neighboring examples from other classes when generating synthetic examples. This could result in more class overlap and noise. This is especially bad if you have a ...


0

Classic evaluation metrics are heavily affected by skewed data. In your case, since you have imbalanced data, you should definitely avoid those such as accuracy. For example, Imagine you have a test data with 100 records. 3 of them have Class A and 97 have Class B. I create this model: prediction = "B" Simple as that, I always give the second ...


0

Your options are: Do nothing with the model. (Then why did you build it?) Use the model, knowing that it works well enough, even though it makes mistakes. (Evaluating if it works well enough is a separate discussion.) Speech recognition software like Siri make mistakes, yet the software remains useful. The alternative is not to have any speech recognition ...


0

In short I would say: stick with the old probably manual process if it suits you better or try to understand why the result is only 85% and improve the model/data quality try some other models, parameters, engineer other features get more and cleaner data or hire a top DataScientist who could help with that issue When deciding if 85% is enough or not, you ...


0

This is where so-called term Baseline comes into play. One needs to have a baseline either a simple model prediction performance (accuracy, precision or recall whatever) set, and try to improve upon it. Or in a more natural way, when available, it is best to have a human baseline. The latter is quite common in industries, but it is more costly to obtain ...


Top 50 recent answers are included