Questions tagged [accuracy]

In data science, accuracy is a measurement used to determine which model is best at describing the underlying patterns of a dataset.

Filter by
Sorted by
Tagged with
98 votes
4 answers
103k views

Advantages of AUC vs standard accuracy

I was starting to look into area under curve(AUC) and am a little confused about its usefulness. When first explained to me, AUC seemed to be a great measure of performance but in my research I've ...
aidankmcl's user avatar
  • 1,083
55 votes
5 answers
32k views

Is it always better to use the whole dataset to train the final model?

A common technique after training, validating and testing the Machine Learning model of preference is to use the complete dataset, including the testing subset, to train a final model to deploy it on, ...
pcko1's user avatar
  • 3,930
41 votes
8 answers
9k views

What would I prefer - an over-fitted model or a less accurate model?

Let's say we have two models trained. And let's say we are looking for good accuracy. The first has an accuracy of 100% on training set and 84% on test set. Clearly over-fitted. The second has an ...
EitanT's user avatar
  • 519
31 votes
1 answer
26k views

What is a LB score in machine learning?

I was going through an article on kaggle blogs. Repeatedly, the author mentions 'LB score' and 'LB fit') as a metric for effectiveness of machine learning (along with cross validation (CV) score). ...
user345394's user avatar
30 votes
4 answers
56k views

macro average and weighted average meaning in classification_report

I use the "classification_report" from from sklearn.metrics import classification_report in order to evaluate the imbalanced binary classification ...
user10296606's user avatar
  • 1,834
25 votes
3 answers
525 views

How do you manage expectations at work?

With all the hoopla around Data Science, Machine Learning, and all the success stories around, there are a lot of both justified, as well as overinflated, expectations from Data Scientists and their ...
22 votes
2 answers
12k views

How to increase accuracy of classifiers?

I am using OpenCV letter_recog.cpp example to experiment on random trees and other classifiers. This example has implementations of six classifiers - random trees, boosting, MLP, kNN, naive Bayes and ...
Mika's user avatar
  • 323
16 votes
1 answer
79k views

Train Accuracy vs Test Accuracy vs Confusion matrix

After I developed my predictive model using Random Forest I get the following metrics: ...
Pedro Alves's user avatar
13 votes
3 answers
6k views

What are the disadvantages of accuracy?

I have been reading about evaluating a model with accuracy only and I have found some disadvantages. Among them, I read that it equates all errors. How could this problem be solved? Maybe assigning ...
PicaR's user avatar
  • 314
13 votes
2 answers
19k views

Validation vs. test vs. training accuracy. Which one should I compare for claiming overfit?

I have read on the several answers here and on the Internet that cross-validation helps to indicate that if the model will generalize well or not and about overfitting. But I am confused that which ...
A.B's user avatar
  • 316
13 votes
1 answer
55k views

How to know if a model is overfitting or underfitting by looking at graph

Just recently got my hands on tensorboard, but can you tell me what features should I look for in the graph (Accuracy and Validation Accuracy) And please do enlighten me about the concept of ...
Nikhil.Nixel's user avatar
12 votes
1 answer
8k views

Balanced Accuracy vs. F1 Score

I've read plenty of online posts with clear explanations about the difference between accuracy and F1 score in a binary classification context. However, when I came across the concept of balanced ...
Ric S's user avatar
  • 257
11 votes
7 answers
46k views

I got 100% accuracy on my test set,is there something wrong?

I got 100% accuracy on my test set when trained using decision tree algorithm.but only got 85% accuracy on random forest Is there something wrong with my model or is decision tree best suited for the ...
Harigovind Valsakumar's user avatar
11 votes
3 answers
9k views

Inverse Relationship Between Precision and Recall

I made some search to learn precision and recall and I saw some graphs represents inverse relationship between precision and recall and I started to think about it to clarify subject. I wonder the ...
tkarahan's user avatar
  • 422
11 votes
3 answers
12k views

Relationship between KS, AUROC, and Gini

Common model validation statistics like the Kolmogorov–Smirnov test (KS), AUROC, and Gini coefficient are all functionally related. However, my question has to do with proving how these are all ...
Steven's user avatar
  • 111
10 votes
5 answers
16k views

In which epoch should i stop the training to avoid overfitting

I'm working on an age estimation project trying to classify a given face in a predefined age range. For that purpose I'm training a deep NN using the keras library. The accuracy for the training and ...
Yiannis Ath's user avatar
10 votes
2 answers
7k views

How to get an aggregate confusion matrix from n different classifications

I want to test the accuracy of a methodology. I ran it ~400 times, and I got a different classification for each run. I also have the ground truth, i.e., the real classification to test against. For ...
gc5's user avatar
  • 879
10 votes
4 answers
8k views

Log loss vs accuracy for deciding between different learning rates?

While model tuning using cross validation and grid search I was plotting the graph of different learning rate against log loss and accuracy separately. Log loss When I used log loss as score in ...
CodeMaster GoGo's user avatar
8 votes
2 answers
227 views

Why isn't dimension sampling used with gradient boosting machines (GBM)?

GBMs, like random forests, build each tree on a different sample of the dataset and hence, going by the spirit of ensemble models, produce higher accuracies. However, I have not seen GBM being used ...
Nitesh's user avatar
  • 1,615
7 votes
3 answers
2k views

0.1 accuracy on MNIST fashion dataset following official Tensorflow/Keras tutorial

My goal is to classify products pictures into categories such as dress, sandals, etc. I am using the MNIST fashion dataset, following this official tutorial word-per-word: https://www.tensorflow.org/...
Nicolas Raoul's user avatar
7 votes
2 answers
2k views

Why validation loss worsens while precision/recall continue to improve?

I'm training a neural network on 'easy' dataset with ~15k examples. Network overfits pretty fast. The thing I cannot understand that after 5th epoch validation loss is starting to worsen, while ...
Vadym B.'s user avatar
  • 251
7 votes
2 answers
208 views

What makes you confident in your results? At what point do you think you can present your work to tech illiterate superiors?

I understand that the models are only as good as the data you get, and bad design can generate really bad data. Nonrandom sampling, unbalanced/incomplete designs, confounding, can make data analysis ...
user2801011's user avatar
7 votes
2 answers
769 views

Coursera ML - Does the choice of optimization algorithm affect the accuracy of multiclass logistic regression?

I recently completed exercise 3 of Andrew Ng's Machine Learning on Coursera using Python. When initially completing parts 1.4 to 1.4.1 of the exercise, I ran into difficulties ensuring that my ...
AKKA's user avatar
  • 123
7 votes
1 answer
1k views

Multiclass classification on imbalanced dataset : Accuracy or micro F1 or macro F1

I have a multiclass classification problem. Further, an instance can be assigned to exactly one class. My dataset is highly imbalanced. I know that accuracy is not a good metric to use in this case ...
Bikash Gyawali's user avatar
6 votes
2 answers
5k views

human level performance on ImageNet, top-1 or top-5?

Anyone have pointers to where the human level performance on ImageNet comes from? I found a reference to 5.1% accuracy (top-1? or top-5?) from here.
fseto's user avatar
  • 163
6 votes
4 answers
23k views

How to improve accuracy of deep neural networks

I am using Tensorflow to predict whether the given sentence is positive and negative. I have take 5000 samples of positive sentences and 5000 samples of negative sentences. 90% of the data I used it ...
deepguy's user avatar
  • 1,441
6 votes
3 answers
8k views

How to determine if my GBM model is overfitting?

Below is a simplified example of a h2o gradient boosting machine model using R's iris dataset. The model is trained to predict sepal length. The example yields an r2 value of 0.93, which seems ...
Borealis's user avatar
  • 347
6 votes
5 answers
26k views

How to compute f1 in TensorFlow

I have a code that computes the accuracy, but now I would like to compute the F1 score. ...
William Scott's user avatar
6 votes
5 answers
24k views

How to improve loss and avoid overfitting

I'm trying to build a 2 class image classifier using the architecture suggested in first part of this blog https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data....
Amit Khanna's user avatar
6 votes
1 answer
7k views

What is the classification accuracy of a random classifier?

I have a build a classification model using machine learning technique (SVM). I want to compare the classification accuracy of my model with a random classifier. My data set contains only two classes(...
Soikot Ali's user avatar
6 votes
3 answers
2k views

Accuracy is lower than f1-score for imbalanced data

For a binary classification, I have a dataset with 55% negative label and 45% positive labels. The results of the classifier shows that the accuracy is lower than the f1-score. Does that mean that the ...
ds_newbie's user avatar
6 votes
2 answers
2k views

Why does degradation occur in deep neural networks?

It has been shown that "plain" neural networks tend to have an increased amount training error, and accompanied test error, as more layers are added. I am not quite certain as to why this occurs. In ...
Sahil Kulkarni's user avatar
6 votes
1 answer
101 views

How to define a custom resampling methodology

I'm using an experimental design to test the robustness of different classification methods, and now I'm searching for the correct definition of such design. I'm creating different subsets of the ...
gc5's user avatar
  • 879
5 votes
3 answers
13k views

Is Gini coefficient a good metric for measuring predictive model performance on highly imbalanced data

I am evaluating a Credit Risk model that predicts the estimated likelihood of customers defaulting on their mortgage accounts. The model is a Logistic Regression estimator and was built by another ...
John's user avatar
  • 53
5 votes
2 answers
575 views

scikit-learn RandomForestClassifier always hits 100% test accuracy

I have been playing with a toy problem to compare the performance and behavior of several scikit-learn classifiers. Brief, I have one continuous variable X (which contains two samples of size N, each ...
Aaron Ponti's user avatar
5 votes
1 answer
6k views

How to choose the right threshold for binary classification?

I am currently working on the titanic dataset from Kaggle. The data set is imbalanced with almost 61.5 % negative and 38.5 positive class. I divided my training dataset into 85% train and 15% ...
Joe's user avatar
  • 75
5 votes
1 answer
2k views

Training and cross validation error curves

I have a graph which plots training datasize on X axis and accuracy on y axis. I plotted the curves using sklearn's learning_curve. It is observed that the accuracy of training dataset decreases but ...
Hima Varsha's user avatar
  • 2,316
5 votes
1 answer
5k views

Why does adding data augmentation decrease training accuracy a tiny bit?

Before data augmentation, my model clearly overfits and hits a 100% training accuracy and a 52% validation accuracy. When only adding data augmentation with Keras, as a regularization technique, it ...
Kralley's user avatar
  • 53
5 votes
2 answers
4k views

Classification Accuracy in Keras

I'm using two different functions to calculate the accuracy of my deep learning model and I am confused which one is which. The first one is ...
gabi's user avatar
  • 159
4 votes
2 answers
4k views

My data is highly overlapping, but when I apply logistic regression, it is giving an impressive accuracy of 79%. Why?

Logistic regression is supposed to work well only on data that is linearly separable. As we can see in the pair plot, the data points heavily overlap. The logistic regression model is in fact showing ...
Apoorva's user avatar
  • 275
4 votes
2 answers
1k views

Why are results without Transfer Learning better than with Transfer Learning?

I developed a neural network for license plate recognition and used the EfficientNet architecture (https://keras.io/api/applications/efficientnet/#efficientnetb0-function) with and without pretrained ...
Tobitor's user avatar
  • 93
4 votes
1 answer
777 views

What does a predicted probability really mean, without considering the accuracy of the underlying model?

Say I've built a (completely unrealistic) classification model in Keras that gives me 1.00 accuracy. And next, I would like to use my model on some new, unseen data, and use ...
Monica Heddneck's user avatar
4 votes
1 answer
528 views

Performance of model in production varying greatly from train-test data

I was wondering if anyone has any advice on where to start digging for this problem. I have a model which has gone through development and all train/cv/test data sets now perform above 95% both for ...
1961DarthVader's user avatar
4 votes
3 answers
149 views

Is my model over fitting or not?

I have 50000 observations with 70% positive and 30% negative target variable. I'm getting accuracy of around 96-99% which seems unreal of course and I'm worried that my model is over-fitting which I ...
hyeri's user avatar
  • 141
3 votes
4 answers
1k views

Why is my test data accuracy higher than my training data?

I'm using four years of data, training on the first 3 and testing on the fourth. Using LSTM w/ Keras. My test data set (which has no overlap at all with the training) is consistently performing better ...
Odj fourth's user avatar
3 votes
2 answers
419 views

How to compare paired count data?

I am working with a machine learning approach that counts cars in images. I have a predicted dataset, which is the predicted output from the machine learning approach and a paired "true" dataset, ...
Borealis's user avatar
  • 347
3 votes
1 answer
7k views

99% validation accuracy but 0% prediction results (UNET Architecture)

I am debugging results from the UNET architecture that I am using for identifying corneal reflection in eye images. While I am getting over 99% training accuracy and also very high (over 99%) ...
codeexplorer123's user avatar
3 votes
3 answers
501 views

Check Accuracy of Model Provided by Consultant

My company has recently engaged a consultant firm to develop a predictive model to detect defective works. I understand that there are many ways to validate the model, for example, using k-fold cross-...
Alex Yu's user avatar
  • 33
3 votes
4 answers
1k views

Metrics to determine K in K-cross fold validation

Consider a scenario where the dataset in hand is quite large, let's assume 50000 samples (quite well balanced between two classes). What metrics can be used to decide the K value in a K-fold cross-...
NCL's user avatar
  • 211
3 votes
2 answers
684 views

Understanding Sklearns learning_curve

I have been using sklearns learning_curve , and there are a few questions I have that are not answered by the documentation(see also here and here), as well as questions that are raised by the ...
Abijah's user avatar
  • 181

1
2 3 4 5
9