35 votes
Accepted

What is a LB score in machine learning?

In the context of Kaggle, it means LeaderBoard (emphasis mine).
Emre's user avatar
  • 10.5k
33 votes

Is it always better to use the whole dataset to train the final model?

A point that needs to be emphasized about statistical machine learning is that there are no guarantees. When you estimate performance using a held-out set, that is just an estimate. Estimates can be ...
D.W.'s user avatar
  • 3,341
26 votes

macro average and weighted average meaning in classification_report

Your data set is unbalanced since 28432 out of 28481 examples belong to class 0 (that is 99.8%). Therefore, your predictor almost always predicts any given sample as belonging to class 0 and thereby ...
Jonathan's user avatar
  • 5,400
25 votes
Accepted

Train Accuracy vs Test Accuracy vs Confusion matrix

Definitions Accuracy: The amount of correct classifications / the total amount of classifications. The train accuracy: The accuracy of a model on examples it was constructed on. The test accuracy ...
S van Balen's user avatar
  • 1,364
23 votes

Is it always better to use the whole dataset to train the final model?

I personally haven't seen that for products going into production, but understand the logic. Theoretically, the more data your deployed model has seen, the better is should generalise. So if you ...
n1k31t4's user avatar
  • 14.8k
21 votes

What would I prefer - an over-fitted model or a less accurate model?

There are a couple of nuances here. Complexity question very important - ocams razor CV - is this trully the case 84%/83% (test it for train+test with CV) Given this, personal opinion: Second one. ...
Noah Weber's user avatar
  • 5,669
21 votes
Accepted

What are the disadvantages of accuracy?

A common complaint about accuracy is that it fails when the classes are imbalanced. For instance, if you get an accuracy of $98\%$, that sounds like a high $\text{A}$ in school, so you might be pretty ...
Dave's user avatar
  • 3,688
20 votes
Accepted

In which epoch should i stop the training to avoid overfitting

As long as your validation accuracy increases, you should keep training. I would stop when the test accuracy starts decreasing (this is known as early stopping). The general advise is always to keep ...
David Masip's user avatar
  • 6,051
19 votes

Is it always better to use the whole dataset to train the final model?

Once you have obtained optimal hyperparamters for your model, after training and cross validating etc., in theory it is ok to train the model on the entire dataset to deploy to production. This will, ...
PyRsquared's user avatar
  • 1,604
15 votes
Accepted

How to know if a model is overfitting or underfitting by looking at graph

Overfitting is a scenario where your model performs well on training data but performs poorly on data not seen during training. This basically means that your model has memorized the training data ...
zachdj's user avatar
  • 2,684
15 votes

What would I prefer - an over-fitted model or a less accurate model?

It depends mostly on the problem context. If predictive performance is all you care about, and you believe the test set to be representative of future unseen data, then the first model is better. (...
Ben Reiniger's user avatar
  • 11.6k
12 votes
Accepted

Balanced Accuracy vs. F1 Score

One major difference is that the F1-score does not care at all about how many negative examples you classified or how many negative examples are in the dataset at all; instead, the balanced accuracy ...
A Kareem's user avatar
  • 823
11 votes

My data is highly overlapping, but when I apply logistic regression, it is giving an impressive accuracy of 79%. Why?

Decision Tree, KNN, & Random Forest (Methods that are suitable for overlapping data) This statement is false. All those methods are good when the decision surface (separating surface) has a ...
rapaio's user avatar
  • 4,723
10 votes
Accepted

Validation vs. test vs. training accuracy. Which one should I compare for claiming overfit?

Which two accuracies I compare to see if the model is overfitting or not? You should compare the training and test accuracies to identify over-fitting. A training accuracy that is subjectively far ...
Esmailian's user avatar
  • 9,252
9 votes

Accuracy for Kmeans clustering

Accuracy is a measure of comparing the true label to the predicted label. K-Means is an unsupervised clustering algorithm where a predicted label does not exist. So, accuracy can not be directly ...
E. Kenney's user avatar
  • 304
9 votes

What would I prefer - an over-fitted model or a less accurate model?

The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted. Maybe not. It's true that 100% training accuracy is usually a strong indicator of overfitting, but it's ...
Ray's user avatar
  • 191
8 votes

99% validation accuracy but 0% prediction results (UNET Architecture)

There is no "mismatch" of accuracy. Your problem is that you have an image segmentation problem where 99% of the pixels should be zero. So getting 99% accuracy is trivially easy. A model that predicts ...
Neil Slater's user avatar
  • 28.8k
8 votes
Accepted

I got 100% accuracy on my test set,is there something wrong?

There may be a few reason this is happening. First of all, check your code. 100% accuracy seems unlikely in any setting. How many testing data points do you have? How many training data points did ...
c zl's user avatar
  • 146
8 votes

What are the disadvantages of accuracy?

In general, the main disadvantage of accuracy is that it masks the issue of class imbalance. For example if the data contains only 10% of positive instances, a majority baseline classifier which ...
Erwan's user avatar
  • 25.2k
7 votes
Accepted

0.1 accuracy on MNIST fashion dataset following official Tensorflow/Keras tutorial

You haven't normalized your image dataset such as setting the pixel values between 0-1 which could help classifier converge faster. Please do it by doing the operation below. ...
Syed Nauyan Rashid's user avatar
7 votes
Accepted

Inverse Relationship Between Precision and Recall

If we decrease the false negative (select more positives), recall always increases, but precision may increase or decrease. Generally, for models better than random, precision and recall have an ...
Esmailian's user avatar
  • 9,252
6 votes
Accepted

How to determine if my GBM model is overfitting?

The term overfitting means the model is learning relationships between attributes that only exist in this specific dataset and do not generalize to new, unseen data. Just by looking at the model ...
Simon Boehm's user avatar
6 votes
Accepted

Is Gini coefficient a good metric for measuring predictive model performance on highly imbalanced data

The Gini Coefficient can also be expressed in terms of the area under the ROC curve (AUC): G = 2*AUC -1 link. The ROC curve, on the other hand, is influenced by ...
oW_'s user avatar
  • 6,327
6 votes
Accepted

human level performance on ImageNet, top-1 or top-5?

It comes from this paper: https://arxiv.org/abs/1409.0575 O. Russakovsky "ImageNet Large Scale Visual Recognition Challenge" 2014
keiv.fly's user avatar
  • 1,239
6 votes

Validation vs. test vs. training accuracy. Which one should I compare for claiming overfit?

Cross validation splits your data into K folds. Each fold contains a set of training data and test data. You are correct that you get K different error rates that you then take the mean of. These ...
astel's user avatar
  • 347
6 votes

loss/val_loss are decreasing but accuracies are the same in LSTM!

I think this is because your targets y are continuous instead of binary. Therefore, either ignore the accuracy report, or ...
Esmailian's user avatar
  • 9,252
6 votes

Is it better to have higher train accuracy with lower test accuracy or higher test accuracy with lower train accuracy?

Test accuracy better reflects generalization error, so you want the one with higher test accuracy. In your first setup, the higher train accuracy indicates overfitting, as it's significantly higher ...
Sean Owen's user avatar
  • 6,595
6 votes

My data is highly overlapping, but when I apply logistic regression, it is giving an impressive accuracy of 79%. Why?

Your data is multidimensional, it is possible that any two dimensional projection overlaps while still existing an hyperplane on the original dimensionality that separates the two classes well Say for ...
Khanis Rok's user avatar
5 votes

How to improve loss and avoid overfitting

I am not familiar with the software you are using but keep in mind: You EXPECT accuracy to drop if you reduce over fitting. It is not a bad thing. Over-fitting is essentially "fake accuracy". Some ...
Julian Kurz's user avatar
5 votes
Accepted

How to improve accuracy of deep neural networks

You need much more data. Deep NN shines when you have excessive amounts of data. With only a little bit if data it can easily overfit. The big difference between training and test performance shows ...
Has QUIT--Anony-Mousse's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible