Questions tagged [accuracy]
In data science, accuracy is a measurement used to determine which model is best at describing the underlying patterns of a dataset.
114
questions with no upvoted or accepted answers
4
votes
2
answers
54
views
evaluation metrics for multiple values per session
I have an application that executes my foo() function several times for each user session. There are 2 alternate algorithms that I can implement as ...
- 141
3
votes
0
answers
790
views
Target mean encoding worse than ordinal encoding with GBDT ( XGBoost, CatBoost )
I have a dataset of 23k rows of an unbalanced dataset 85/15 ratio, 10 variables ( 9 of which are categorical ) , i'm using CatBoost and XGBoost for a binary classification.
I applied cv (5 iteration ...
- 2,094
3
votes
1
answer
366
views
How to get the number of steps until a certain accuracy in keras?
I want to see how many steps does it take for my model to reach a certain accuracy.Say 90 percent on cifar10.How can I get this info from the keras model ?
EDIT:
accuracy in each epoch is accessible ...
- 221
3
votes
0
answers
525
views
Validation score (f1) remains the same when swapping labels
I have an imbalanced dataset (True labels are ~10x than False labels) and thus use the f_beta score as a metric for model performance, as such:
...
- 133
2
votes
0
answers
77
views
How much ground truth is needed for a classification model?
I have unstructured problem text that needs to be classified into categories(Multinomial classification). Depending on the component, which is a structured element that allows me to segment the data, ...
- 21
2
votes
0
answers
109
views
how to improve recall by retraining a model on its feedback
I am creating a supervised model using sensitive and scarce data. For the sake of discussion, I've simiplified the problem statement by assuming that I'm creating a model for identifying dogs.
Let's ...
2
votes
1
answer
48
views
Query regarding surprising spike in accuracy of ML model
I implemented all the major ML models (Logistic Regression, Naive Bayes, SVM, KNN, Decision Tree, Random Forest, Ada Boost & XGBoost) on my dataset. My stratified cross-validation scores are ...
- 317
2
votes
1
answer
354
views
what are the next step after ML prediction and how to proceed?
I have trained an ML model with a good accuracy but what next?
I am facing difficulty in answering this question, how will you present your model.
Which framework do you use
How do you make sure ...
- 87
2
votes
1
answer
1k
views
Metrics for presenting RNN/LSTM result
I am working on two different architectures based on the LSTM model to predict the user's next action based on the previous actions. I am wondering, what is the best way to present the result? Is it ...
2
votes
1
answer
1k
views
How do I perform Leave One Out Cross Validation For Top n Recommendation Sytems?
I am new in making recommendation systems . I am using the surpriselib library to evaluate my recommendations. All the Accuracy Metrics are well supported in this library. But I also want to compute ...
- 21
2
votes
2
answers
716
views
Generalization of accuracy score based on subset of data points
I have a multi-class problem that I am building a classifier for. I have N total data points I would like to predict on. If I instead predict on n < N data points and get an accuracy score, is ...
- 141
1
vote
0
answers
24
views
Validation Accuracy incorrect when multiple outputs are in the dense layer!
I have a set-up a parallel LSTM architecture with two LSTM layers and one Dense layer producing several outputs which are converted into a probability with a sigmoid function on the dense layer. For ...
- 11
1
vote
1
answer
34
views
Can we train of a binary classifier with "A" to classify "a"?
I have a maybe naive question about the appropriateness of using binary classifications. This is a hypothetical example, so forgive me if it is too coarse.
Let's say I want to train a support vector ...
- 11
1
vote
0
answers
114
views
Error from XGBoost missing data handling
I have a regression problem with a very large dataset >50 million rows, 81 features and 1 target, all positive float values unevenly distributed between 0 - 1 million. I've trained an XGBoost model ...
- 36
1
vote
0
answers
12
views
How do I test one-shot model preformance against flawed categories?
I'm in the process of reworking the ASAM database. Excerpted, it looks like this:
...
- 131
1
vote
0
answers
230
views
What is mean accuracy and why is it a harsh metric for multi-label validation?
The score method docs for scikit-learn's SGDClassifier have the following description:
Return the mean accuracy on the given test data and labels.
In multi-label classification, this is the subset ...
- 215
1
vote
2
answers
2k
views
model.fit vs model.evaluate gives different results?
The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or ...
- 111
1
vote
0
answers
18
views
Comparing two models with different (naive) baseline
I would like to compare a model with listwise deletion to a model with multiple imputation. However, the model with listwise deletion has a majority class of 70%, while the model with multiple ...
- 11
1
vote
0
answers
89
views
how to train image classification with small difference in low-level feature
I would like to build the model that capable of doing classification for example 2 classes below :
I tried alexnet, resnet50, resnet18, vgg16 but seem they are failed to differentiate between this ...
- 11
1
vote
0
answers
73
views
How can I calculate the level of agreement between my K-Means cluster labels and my ground truth labels in R?
I have made a K-Means clustering from 3 rasters with various values of k (k=2, k=4, k=7) and would like to know which values of k explains the most variance in my ground-truth data or the value of k ...
- 11
1
vote
1
answer
302
views
Why would the accuracy of a model change when the loss doesn't?
I've trained 8 models based on the same architecture (convolutional neural network), and each uses a different data augmentation method. The accuracy of the models fluctuates greatly while the loss ...
- 111
1
vote
0
answers
37
views
Model Performance is Fluctuating
I am training a 3D u-net model for 3D medical images. My training data has 800 images, the validation data has 200, and test data has 200 images. when I try to fit the model, there is a fluctuation in ...
- 36
1
vote
0
answers
21
views
How to improve model accuracy by redistributing training data over test, validation, and training data subsets after training
How do I improve model accuracy by redistributing training data over test, validation, and training data subsets after training, and then training a new model with the updated data subsets.
The model ...
- 11
1
vote
1
answer
722
views
How to compute f1_score for multiclass multilabel classification
I have used one hot encoder [1,0,0][0,1,0][0,0,1] for my functional classification model.
The predicted probabilities for test data ...
- 151
1
vote
0
answers
20
views
Compute similiarty between labels
I have a labeled dataset and I created a duplicate of this dataset and removed the labels and applied K-means clustering with k= the number of labels in the original data set I want to compute ...
- 137
1
vote
0
answers
91
views
How to calculate separate binary accuracy metric for each label in a multi-label classification in Keras?
I'm stuck with this:
def multilabel_binary_acc(y_true, y_pred):
return [K.equal(y_true[:,n], K.round(y_pred[:,n])) for n in range(y_pred.shape[1])]
But it ...
- 11
1
vote
0
answers
25
views
different scored probability distributions for a classification model using python vs Azure ML Studio
I have a classification model which I initially built in Azure ML studio and then created a similar model in python (its similar because the algorithms in python VS Azure are a bit different so they ...
- 69
1
vote
0
answers
38
views
Tensorboard reported accuracy does not match my own calculations
I'm using Google's AI Platform to train an image classifier with Tensorflow (resnet-50 model). I have only two classes. Training set is something like 4K images in size and validation set 1K images. ...
- 131
1
vote
0
answers
34
views
Why does my model fail to predict on the whole dataset?
So I have about 3000 images with 6 classes and this is what I did:
1 - split into training set and test set prior to anything with 20% test size
2 - performed data augmentation on the under ...
- 21
1
vote
0
answers
29
views
Improving Training accuracy of LSTM in Keras for ratings prediction given reviews
I'm new to Deep Learning and NLP. I found a dataset online which has reviews of different companies and their corresponding ratings from 1 to 5. I encoded the labels, then removed some basic stop ...
- 11
1
vote
0
answers
202
views
Why accuracy scores reported by Keras are low and erratic while the loss on the validation set is decreasing?
I'm trying to build CNN to predict two-label classification problem. Unfortuenetely, I can't share my model architecture, but I compiled the model using:
...
- 95
1
vote
0
answers
330
views
g-mean for binary classification doesn't use sensitivity of each class?
scikit-learn's contrib package, imbalanced-learn, has a function, geometric_mean_score(), ...
- 111
1
vote
0
answers
111
views
jumpy validation accuracy graph
helo,
i have a very weird problem.
my validation accuracy graph is very jumpy, and i dont know how to fix it
this is the graph:
this is a multi label problem. i calculate accuracy with 0.5 threshold
...
- 41
1
vote
0
answers
78
views
Since is not possible test all the possible combination, what is the correct procedure to follow on building Machine Learning?
Sorry, I'm a little confused and this is a general question. How can I be sure that the procedure that I am following is the correct one? Following the steps for building a machine learning model, we ...
- 11
1
vote
0
answers
46
views
Classification Model showing different accuracy for SAME data?
This is my first post here, so kindly pardon any commonplace errors.
So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each ...
- 111
1
vote
0
answers
63
views
What is mAP in object detection?
I have been reading through this blog in order to find what mAP is .In the sub heading of AP, they give the example of 5 apple images and finding out the average precision.As far I understand false ...
- 141
1
vote
2
answers
673
views
Same validation accuracy, different train accuracy for two neural networks models
I'm performing emotion classification over FER2013 dataset. I'm trying to measure different models performance, and when I checked ImageDataGenerator with a model I had already used I came up with the ...
- 111
1
vote
0
answers
680
views
Pytorch testing/validation accuracy over 100%
So I was training my CNN for some hours when it reached 99% accuracy (which was a little bit too good, I thought). But then it didn´t stop and it went higher than 100%. So I thought, that must be ...
- 133
1
vote
0
answers
27
views
Binary training result in chainer
I am training simple Chainer based CNN to recognise MNIST samples. To each sample I add poissonian noise. For the test purpose I have always the same random seed. I restart training resetting the ...
1
vote
0
answers
89
views
30% accuracy for training set, 80% for test set with a 0.3 split
I have a time series dataset on which I am training. For some reason, the training accuracy is 30% while the test accuracy is about 88% after about 10 epochs. Is this at all normal?
I should point ...
- 111
1
vote
0
answers
70
views
Different accuracy values using the same saved model in tensorflow
I have trained a model in Tensorflow (for some signal classification problem, using mostly convolutional layers, no RNNs), saved It using the callback checkpoints. When I'm testing the said model on a ...
- 11
1
vote
0
answers
27
views
Different testing and training accuracy values within a NN TensorFlow structure
In order to select the optimum number of my gradient descent algorithm, I had used a for loop of 1500 iterations and each 100 iterations training and testing accuracies are printed. Here everything is ...
- 165
1
vote
0
answers
229
views
Regression model Giving the same prediction for all new inputs until i load the model again
I have build a regression model that has some decent accuracy measures.
I have pickled it and loading it another project. However it is producing the same predictions every time when i pass new ...
- 111
1
vote
1
answer
290
views
My accuracy changes throughout every epoc but the val_acc at the end of each epoc stays the same
I am training a transfer learning CNN on 161 pictures that have been augmented into 966 photos, 4 classes. I am training on a balanced data set, so there are 52 images of each class and also in the ...
- 81
1
vote
0
answers
35
views
Is it possible to have a better performance with C4.5 than Bagged tree?
I am not sure but I have read that Bagged tree is used to improve the accuracy of a signle tree methods such as C4.5 but applying both of them over the same dataset I got better accuracy with C4.5, ...
- 11
1
vote
1
answer
943
views
test accuracy of text classification is too less
I have a data set of movies and their subtitles. My task is to classify them based on their ratings - [R, NR, PG, PG-13, G].
I have 13 examples for each class.
I preprocessed the subtitles in the ...
- 117
1
vote
0
answers
22
views
Validating performance of panel data based models
I'm wondering from a theoretical/general practice perspective, what is the best way to evaluate performance of regression models derived from panel data (i.e. a time series of cross sectional data). ...
- 111
1
vote
1
answer
73
views
Optimize F-Score only for certain classes, disregard other classes
I have a labeled dataset of product reviews where the label is a rating between 1 and 5 and the review is just text.
I use a simple naive Bayes classifier (sklearn) to try to predict a rating given a ...
- 111
1
vote
0
answers
784
views
Prediction in the training sample with randomforest in r
I'm using a Random Forest algorithm in order to construct a classification model, and I HAVE to check the accuracy of my rf model in the training sample, but as you can see in this answers :
https://...
- 83
1
vote
0
answers
36
views
Making sense of blocks python package output
I've modified the Blocks tutorial, to train an MLP neural net with a dataset provided for an assignment in my ML course.
I'd like to evaluate the accuracy of the ...
- 111