Questions tagged [accuracy]

In data science, accuracy is a measurement used to determine which model is best at describing the underlying patterns of a dataset.

124 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
2 answers
44 views

evaluation metrics for multiple values per session

I have an application that executes my foo() function several times for each user session. There are 2 alternate algorithms that i can implement as "foo" function and my goal is to evaluate them based ...
sbr's user avatar
  • 131
3 votes
0 answers
768 views

Target mean encoding worse than ordinal encoding with GBDT ( XGBoost, CatBoost )

I have a dataset of 23k rows of an unbalanced dataset 85/15 ratio, 10 variables ( 9 of which are categorical ) , i'm using CatBoost and XGBoost for a binary classification. I applied cv (5 iteration ...
Blenz's user avatar
  • 2,054
3 votes
1 answer
352 views

How to get the number of steps until a certain accuracy in keras?

I want to see how many steps does it take for my model to reach a certain accuracy.Say 90 percent on cifar10.How can I get this info from the keras model ? EDIT: accuracy in each epoch is accessible ...
Moeinh77's user avatar
  • 221
3 votes
0 answers
521 views

Validation score (f1) remains the same when swapping labels

I have an imbalanced dataset (True labels are ~10x than False labels) and thus use the f_beta score as a metric for model performance, as such: ...
Jens de Bruijn's user avatar
2 votes
0 answers
54 views

How much ground truth is needed for a classification model?

I have unstructured problem text that needs to be classified into categories(Multinomial classification). Depending on the component, which is a structured element that allows me to segment the data, ...
Ozzie's user avatar
  • 21
2 votes
0 answers
88 views

how to improve recall by retraining a model on its feedback

I am creating a supervised model using sensitive and scarce data. For the sake of discussion, I've simiplified the problem statement by assuming that I'm creating a model for identifying dogs. Let's ...
learnlifelong's user avatar
2 votes
1 answer
230 views

what are the next step after ML prediction and how to proceed?

I have trained an ML model with a good accuracy but what next? I am facing difficulty in answering this question, how will you present your model. Which framework do you use How do you make sure ...
yathislax's user avatar
2 votes
3 answers
1k views

Manual way to draw accuracy/loss graphs

During the training process of the convolutional neural network, the network outputs the training/validation accuracy/loss after each epoch as shown below: ...
Ali Raza Memon's user avatar
2 votes
1 answer
48 views

Accuracy after selftraining didn't change

I used Decisiton Tree Classifier which I trained with 50 000 samples. I have also set with unlabeled samples, so I decided to use self training algorithm. Unlabeled set has 10 000 samples. I would ...
SMI9's user avatar
  • 21
2 votes
1 answer
1k views

Metrics for presenting RNN/LSTM result

I am working on two different architectures based on the LSTM model to predict the user's next action based on the previous actions. I am wondering, what is the best way to present the result? Is it ...
Bloodstone Programmer's user avatar
2 votes
1 answer
1k views

How do I perform Leave One Out Cross Validation For Top n Recommendation Sytems?

I am new in making recommendation systems . I am using the surpriselib library to evaluate my recommendations. All the Accuracy Metrics are well supported in this library. But I also want to compute ...
Saket Aryan's user avatar
2 votes
2 answers
654 views

Generalization of accuracy score based on subset of data points

I have a multi-class problem that I am building a classifier for. I have N total data points I would like to predict on. If I instead predict on n < N data points and get an accuracy score, is ...
Jake Morris's user avatar
2 votes
2 answers
1k views

Ideal difference in the training accuracy and testing accuracy

In a data classification problem (with supervised learning), what should be the ideal difference in the training set accuracy and testing set accuracy? What should be the ideal range? Is a difference ...
girl101's user avatar
  • 1,161
1 vote
0 answers
23 views

Validation Accuracy incorrect when multiple outputs are in the dense layer!

I have a set-up a parallel LSTM architecture with two LSTM layers and one Dense layer producing several outputs which are converted into a probability with a sigmoid function on the dense layer. For ...
Abdi's user avatar
  • 11
1 vote
1 answer
34 views

Can we train of a binary classifier with "A" to classify "a"?

I have a maybe naive question about the appropriateness of using binary classifications. This is a hypothetical example, so forgive me if it is too coarse. Let's say I want to train a support vector ...
Patrick's user avatar
  • 11
1 vote
1 answer
58 views

Balancing between accuracy and model overfitting

I have a dataset and I have built an XGBClassifier model from it. Without hyperparameter tuning, the model performs fairly well in training but on test which have some signs of overfitting (train ...
Kusisi Karem's user avatar
1 vote
0 answers
98 views

Error from XGBoost missing data handling

I have a regression problem with a very large dataset >50 million rows, 81 features and 1 target, all positive float values unevenly distributed between 0 - 1 million. I've trained an XGBoost model ...
lexan55's user avatar
  • 36
1 vote
0 answers
12 views

How do I test one-shot model preformance against flawed categories?

I'm in the process of reworking the ASAM database. Excerpted, it looks like this: ...
hrokr's user avatar
  • 131
1 vote
0 answers
201 views

What is mean accuracy and why is it a harsh metric for multi-label validation?

The score method docs for scikit-learn's SGDClassifier have the following description: Return the mean accuracy on the given test data and labels. In multi-label classification, this is the subset ...
Chris Snow's user avatar
1 vote
2 answers
2k views

model.fit vs model.evaluate gives different results?

The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or ...
ptn77's user avatar
  • 111
1 vote
0 answers
18 views

Comparing two models with different (naive) baseline

I would like to compare a model with listwise deletion to a model with multiple imputation. However, the model with listwise deletion has a majority class of 70%, while the model with multiple ...
Tessa's user avatar
  • 11
1 vote
0 answers
78 views

how to train image classification with small difference in low-level feature

I would like to build the model that capable of doing classification for example 2 classes below : I tried alexnet, resnet50, resnet18, vgg16 but seem they are failed to differentiate between this ...
Aryo Wibowo's user avatar
1 vote
1 answer
45 views

Query regarding surprising spike in accuracy of ML model

I implemented all the major ML models (Logistic Regression, Naive Bayes, SVM, KNN, Decision Tree, Random Forest, Ada Boost & XGBoost) on my dataset. My stratified cross-validation scores are ...
Apoorva's user avatar
  • 275
1 vote
0 answers
71 views

How can I calculate the level of agreement between my K-Means cluster labels and my ground truth labels in R?

I have made a K-Means clustering from 3 rasters with various values of k (k=2, k=4, k=7) and would like to know which values of k explains the most variance in my ground-truth data or the value of k ...
R.runya's user avatar
  • 11
1 vote
1 answer
239 views

Why would the accuracy of a model change when the loss doesn't?

I've trained 8 models based on the same architecture (convolutional neural network), and each uses a different data augmentation method. The accuracy of the models fluctuates greatly while the loss ...
NominalSystems's user avatar
1 vote
0 answers
35 views

Model Performance is Fluctuating

I am training a 3D u-net model for 3D medical images. My training data has 800 images, the validation data has 200, and test data has 200 images. when I try to fit the model, there is a fluctuation in ...
User's user avatar
  • 36
1 vote
0 answers
21 views

How to improve model accuracy by redistributing training data over test, validation, and training data subsets after training

How do I improve model accuracy by redistributing training data over test, validation, and training data subsets after training, and then training a new model with the updated data subsets. The model ...
ahk3's user avatar
  • 11
1 vote
1 answer
565 views

How to compute f1_score for multiclass multilabel classification

I have used one hot encoder [1,0,0][0,1,0][0,0,1] for my functional classification model. The predicted probabilities for test data ...
Kyv's user avatar
  • 151
1 vote
0 answers
20 views

Compute similiarty between labels

I have a labeled dataset and I created a duplicate of this dataset and removed the labels and applied K-means clustering with k= the number of labels in the original data set I want to compute ...
Mohamed Amine's user avatar
1 vote
0 answers
90 views

How to calculate separate binary accuracy metric for each label in a multi-label classification in Keras?

I'm stuck with this: def multilabel_binary_acc(y_true, y_pred): return [K.equal(y_true[:,n], K.round(y_pred[:,n])) for n in range(y_pred.shape[1])] But it ...
Al777's user avatar
  • 11
1 vote
0 answers
25 views

different scored probability distributions for a classification model using python vs Azure ML Studio

I have a classification model which I initially built in Azure ML studio and then created a similar model in python (its similar because the algorithms in python VS Azure are a bit different so they ...
Fatima's user avatar
  • 71
1 vote
0 answers
38 views

Tensorboard reported accuracy does not match my own calculations

I'm using Google's AI Platform to train an image classifier with Tensorflow (resnet-50 model). I have only two classes. Training set is something like 4K images in size and validation set 1K images. ...
Patrick's user avatar
  • 131
1 vote
0 answers
31 views

Why does my model fail to predict on the whole dataset?

So I have about 3000 images with 6 classes and this is what I did: 1 - split into training set and test set prior to anything with 20% test size 2 - performed data augmentation on the under ...
Marco Ramos's user avatar
1 vote
0 answers
29 views

Improving Training accuracy of LSTM in Keras for ratings prediction given reviews

I'm new to Deep Learning and NLP. I found a dataset online which has reviews of different companies and their corresponding ratings from 1 to 5. I encoded the labels, then removed some basic stop ...
Sree696's user avatar
  • 11
1 vote
0 answers
192 views

Why accuracy scores reported by Keras are low and erratic while the loss on the validation set is decreasing?

I'm trying to build CNN to predict two-label classification problem. Unfortuenetely, I can't share my model architecture, but I compiled the model using: ...
jakes's user avatar
  • 95
1 vote
2 answers
786 views

Tensorflow keras fit - accuracy and loss both increasing drastically

ubuntu - 20.04 tensorflow 2.2 dataset used = MNIST I am testing tensorflow and i notice that validation sparse_categorical_accuracy (accuracy) and validation <...
figs_and_nuts's user avatar
1 vote
0 answers
300 views

g-mean for binary classification doesn't use sensitivity of each class?

scikit-learn's contrib package, imbalanced-learn, has a function, geometric_mean_score(), ...
Anders Swanson's user avatar
1 vote
0 answers
106 views

jumpy validation accuracy graph

helo, i have a very weird problem. my validation accuracy graph is very jumpy, and i dont know how to fix it this is the graph: this is a multi label problem. i calculate accuracy with 0.5 threshold ...
user2974655's user avatar
1 vote
0 answers
77 views

Since is not possible test all the possible combination, what is the correct procedure to follow on building Machine Learning?

Sorry, I'm a little confused and this is a general question. How can I be sure that the procedure that I am following is the correct one? Following the steps for building a machine learning model, we ...
David's user avatar
  • 11
1 vote
0 answers
44 views

Classification Model showing different accuracy for SAME data?

This is my first post here, so kindly pardon any commonplace errors. So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each ...
Muhammad Yasir's user avatar
1 vote
0 answers
60 views

What is mAP in object detection?

I have been reading through this blog in order to find what mAP is .In the sub heading of AP, they give the example of 5 apple images and finding out the average precision.As far I understand false ...
Fasty's user avatar
  • 141
1 vote
2 answers
594 views

Same validation accuracy, different train accuracy for two neural networks models

I'm performing emotion classification over FER2013 dataset. I'm trying to measure different models performance, and when I checked ImageDataGenerator with a model I had already used I came up with the ...
sneaky_lobster's user avatar
1 vote
0 answers
667 views

Pytorch testing/validation accuracy over 100%

So I was training my CNN for some hours when it reached 99% accuracy (which was a little bit too good, I thought). But then it didn´t stop and it went higher than 100%. So I thought, that must be ...
T Piper's user avatar
  • 123
1 vote
0 answers
27 views

Binary training result in chainer

I am training simple Chainer based CNN to recognise MNIST samples. To each sample I add poissonian noise. For the test purpose I have always the same random seed. I restart training resetting the ...
Lech Wiktor Piotrowski's user avatar
1 vote
0 answers
77 views

30% accuracy for training set, 80% for test set with a 0.3 split

I have a time series dataset on which I am training. For some reason, the training accuracy is 30% while the test accuracy is about 88% after about 10 epochs. Is this at all normal? I should point ...
partoa's user avatar
  • 111
1 vote
0 answers
70 views

Different accuracy values using the same saved model in tensorflow

I have trained a model in Tensorflow (for some signal classification problem, using mostly convolutional layers, no RNNs), saved It using the callback checkpoints. When I'm testing the said model on a ...
DACUS's user avatar
  • 11
1 vote
0 answers
27 views

Different testing and training accuracy values within a NN TensorFlow structure

In order to select the optimum number of my gradient descent algorithm, I had used a for loop of 1500 iterations and each 100 iterations training and testing accuracies are printed. Here everything is ...
baddy's user avatar
  • 165
1 vote
0 answers
221 views

Regression model Giving the same prediction for all new inputs until i load the model again

I have build a regression model that has some decent accuracy measures. I have pickled it and loading it another project. However it is producing the same predictions every time when i pass new ...
Ravi kumar's user avatar
1 vote
1 answer
266 views

My accuracy changes throughout every epoc but the val_acc at the end of each epoc stays the same

I am training a transfer learning CNN on 161 pictures that have been augmented into 966 photos, 4 classes. I am training on a balanced data set, so there are 52 images of each class and also in the ...
Keren's user avatar
  • 81
1 vote
0 answers
35 views

Is it possible to have a better performance with C4.5 than Bagged tree?

I am not sure but I have read that Bagged tree is used to improve the accuracy of a signle tree methods such as C4.5 but applying both of them over the same dataset I got better accuracy with C4.5, ...
MakBad's user avatar
  • 11