Skip to main content
3 votes

Is it bad to average several MAEs calculated from chunks of a big test dataset?

Can I save the MAE from each chunk of data and then average them ? Yes. This is perfectly fine. Why? Think about the metric's definition. Caveat: We assume $k$ chunks of equal chunk size $cs$. If ...
J_H's user avatar
  • 1,130
2 votes

Implementation of spBLEU

spBLEU was introduced in the Flores-101 article: [...] we propose to use BLEU over text tokenized with a single language-agnostic and publicly available fixed SentencePiece subword model. We call ...
noe's user avatar
  • 27k
2 votes

MAD vs RMSE vs MAE vs MSLE vs R²: When to use which?

I recommend that you change your point of view a bit: first decide on what forecast you want, then choose an appropriate error measure. Let me explain. Suppose we are not directly predicting a single ...
Stephan Kolassa's user avatar
2 votes

When is Recall@k useful for a classifier with softmax-like output?

If your use case is producing probabilities of 3 classes, you should use multiclass precision/recall/f1, rather than the @k versions. ...
Karl's user avatar
  • 756
1 vote

Different accuracy scores with sklearn roc_auc_score on same model using sklearn.metrics

There is a fundamental difference between .predict() and .predict_proba(). The former does ...
Luca Anzalone's user avatar
1 vote

Best metrics to evaluate the performance of a regression model?

Mostly depends on the business problem statement & how you want to convey your results. Business folks would like MAPE: Mean absolute percentage error, this atleast gives you how good your model ...
Narahari B M's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible