Questions tagged [accuracy]
In data science, accuracy is a measurement used to determine which model is best at describing the underlying patterns of a dataset.
354
questions
0
votes
1
answer
41
views
Improve true negative of a model
Apart from class balancing (oversampling, undersampling, SMOTE) and hyperparameter tuning, what are other methods that can be used to enhance a True Negative of a model for unbalanced binary target ...
- 41
1
vote
1
answer
36
views
Balancing between accuracy and model overfitting
I have a dataset and I have built an XGBClassifier model from it. Without hyperparameter tuning, the model performs fairly well in training but on test which have some signs of overfitting (train ...
- 41
1
vote
0
answers
15
views
What's the difference between micro-averaged precision and accuracy score?
I'm using sklearn's metrics module to try and evaluate a k-NN model's performance on the provided iris dataset from the ...
- 111
1
vote
0
answers
20
views
Error from XGBoost missing data handling
I have a regression problem with a very large dataset >50 million rows, 81 features and 1 target, all positive float values unevenly distributed between 0 - 1 million. I've trained an XGBoost model ...
- 21
0
votes
0
answers
9
views
Are these ANN training graph and validation graphs incorrect?
I have trained an ANN using Keras (Python3). However, I do not understand the training and validation loss graph. There's a big difference between the first and second training point.
Is the graph of ...
- 21
1
vote
1
answer
25
views
When to prioritize accuracy over precision?
I am working on a simple SVM project for the prediction of hepatitis c. I got my dataset from kaggle. When dealing with null values, I tried two ways, firstly by dropping data with null values, second ...
- 11
0
votes
1
answer
33
views
Why do I get an almost perfect fit as well as bias variance tradeoff with my time series forecast?
In order to achieve scalable and robust time series forecast models, I am currently experimenting with metalearner ensembles.
Note, that I am also using a global modeling approach, so all time series ...
- 145
1
vote
0
answers
10
views
How do I test one-shot model preformance against flawed categories?
I'm in the process of reworking the ASAM database. Excerpted, it looks like this:
...
- 131
0
votes
0
answers
9
views
Orange v3.32: Accuracy and precision not showing up
As explained in the orangehelp files the test and score widget would provide an accuracy colum like "CA". I only have MSE RMSE MAE and R2 besides the times. Furthermore, the predictions ...
- 1
1
vote
1
answer
69
views
How to add class labels to confusion matrix of multi class classification
How do I add class labels to the confusion matrix?
The label display number in the label not the actual value of the label Eg. labels = ['A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P',...
- 23
0
votes
3
answers
76
views
How to verify if the behavior of CNN model is correct?
I am exploring using CNNs for multi-class classification. My model details are:
and the training/testing accuracy/loss:
As you can see from the image, the accuracy jumped from 0.08 to 0.39 to 0.77 ...
- 63
1
vote
0
answers
26
views
What is mean accuracy and why is it a harsh metric for multi-label validation?
The score method docs for scikit-learn's SGDClassifier have the following description:
Return the mean accuracy on the given test data and labels.
In multi-label classification, this is the subset ...
- 215
0
votes
0
answers
9
views
Is it possible for the (Cross Entropy) test loss to increase for a few epochs while the test accuracy also increases?
I came across the question stated in the title:
When training a model with the cross-entropy loss function, is it
possible for the test loss to increase for a few epochs while the
test accuracy also ...
- 1
1
vote
1
answer
48
views
Deep Learning accuracy vs Confusion Matrix accuracy
I am working on deep learning with fer2013 dataset.
After training the model I got val_precision: 0.9168 (precision: 0.8492)
...
- 111
2
votes
0
answers
26
views
How much ground truth is needed for a classification model?
I have unstructured problem text that needs to be classified into categories(Multinomial classification). Depending on the component, which is a structured element that allows me to segment the data, ...
- 21
12
votes
3
answers
4k
views
What are the disadvantages of accuracy?
I have been reading about evaluating a model with accuracy only and I have found some disadvantages. Among them, I read that it equates all errors. How could this problem be solved? Maybe assigning ...
- 302
0
votes
1
answer
37
views
What does it mean if the validation accuracy is equal to the testing accuracy?
I am training a CNN model for my specific problem. I have divided the dataset into 70% training set, 20% validation set, and 10% test set. The validation accuracy achieved was 95% and the test ...
- 13
0
votes
1
answer
14
views
output F1-score instead of Accuracy
I have the code below outputting the accuracy. How can I output the F1-score instead?
...
0
votes
1
answer
27
views
Why my models have a pretty high accuracy with a small training dataset?
I was wondering why my models (decision tree, svm, random forest) behave like that, with "high" accuracy on a small training dataset. Is it a sign of overfitting?
The graph represents the ...
0
votes
1
answer
195
views
model.fit vs model.evaluate gives different results?
The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or ...
- 101
0
votes
0
answers
123
views
How to improve the accuracy of support vector machine algorithms in machine learning?
I am working with a machine learning project named "Diabetes prediction using support vector machine". In this project I have used Pima Indians Diabetes Database. Using SVM I have got 78% ...
0
votes
1
answer
35
views
Random forest accuracy
I saw this video and I understood that to build a random forest are used different decision tree, with a different structure. My code about that is:
...
- 459
2
votes
0
answers
43
views
how to improve recall by retraining a model on its feedback
I am creating a supervised model using sensitive and scarce data. For the sake of discussion, I've simiplified the problem statement by assuming that I'm creating a model for identifying dogs.
Let's ...
1
vote
1
answer
12
views
Does thereshold of classifier close to 0 make sense?
I have roc curve with AUC of 0.91. I applied the following function to determine the best threshold: ...
- 13
1
vote
1
answer
95
views
Explain MAAPE (Mean Arctangent Absolute Percentage Error) in simple terms (intermittent demand forecasting)
n order to measure the accuracy of highly intermitted demand time series, I recently discovered a new accuracy measure, that overcomes the problem of zero values and values close to zero, when ...
- 145
1
vote
1
answer
25
views
How to interpret a Regression Error Characteristic curve
How can I interpret the REC (Regression Error Characteristic) curve ? What is error tolerance and what is the area over the curve? What should be the accepted value for the error tolerance?
How to ...
- 13
0
votes
1
answer
27
views
Scikit learn ComplementNB is outputting NaN for scores
I have an unbalanced binary dataset with 23 features, 92000 rows are labeled 0, and 207,000 rows are labeled 1.
I trained models on this dataset such as GaussianNB, DecisionTreeClassifier, and a few ...
1
vote
1
answer
45
views
Accuracy over 100%
I am using OpenFL, the Intel framework for Federated Learning.
If I run their tutorial example, I have that loss decreases and accuracy is in range 0-100%, like this:
...
- 57
0
votes
0
answers
30
views
Logistic Regression test accuracy vs deployment
I am working on a problem where I make some weekly predictions. I gathered the data myself and did some pre-processing steps and I end up with 6 features. I split the dataset 60-20-20 in train, ...
-1
votes
1
answer
33
views
How to show combined overall accuracy for a multi-ouput model in Keras?
I have a model of the following structure. It has 6 outputs. Given an image, the model predicts classes of 6 different components from the image.
The metrics I used are:
As you can see it outputs an ...
- 1
-1
votes
1
answer
26
views
Difference between the different measurement metric [closed]
Can someone explain what each of these mean? both in simple terms and in terms of TP, TN, FP, FN?
Also are there any other common metrics that I am missing?
F-measure or F-score
Recall
Precision
...
- 3
0
votes
0
answers
4
views
The effects of Double Logarithms (Log Cross Entropy Loss) + Overfitting
My network involves two losses: one is a binary cross entropy, and the other is a multi-label cross entropy.
The yellow graphs are the ones with double logarithm, meaning that we log(sum(ce_loss)). ...
- 1
0
votes
0
answers
16
views
Drop in Accuracy and Precision of a Classifier on Validation set?
We have seen a significant drop in Accuracy by 20% and in Precision by 45%, after running Classifier on the validation set.
During training, we had used 5cv and throughout our tests, we haven't seen ...
- 21
0
votes
0
answers
26
views
Why does my CNN program improve neither the training accuracy nor the validation accuracy despite the error function drastically decreasing?
I have written a Python code to model a convolutional neural network (pastebin link) from the most basic Python libraries (numpy and math in addition to sklearn and pandas being used only for reading ...
1
vote
0
answers
9
views
Comparing two models with different (naive) baseline
I would like to compare a model with listwise deletion to a model with multiple imputation. However, the model with listwise deletion has a majority class of 70%, while the model with multiple ...
- 11
0
votes
1
answer
55
views
resnet50 based model classification loss increase
I'm trying to classify fonts in images into 7 classes.
I wanted to use a pre-trained ResNet50 for the task and use its features to my classification. So I've followed some guide and came up with the ...
- 101
0
votes
1
answer
409
views
How to calculate a single accuracy for a model with multiple outputs in Keras?
Consider the following, rather simple, model:
...
- 257
0
votes
0
answers
19
views
Why are the Order Of Initial Centroids effecting Kmeans Clustering?
For Iris Dataset I am doing the experiment.
iris_k_mean_model_vor = KMeans(n_clusters=3, init=arr_4d)
this is my model. Here I am feeding an Initial array of ...
- 133
1
vote
0
answers
23
views
how to train image classification with small difference in low-level feature
I would like to build the model that capable of doing classification for example 2 classes below :
I tried alexnet, resnet50, resnet18, vgg16 but seem they are failed to differentiate between this ...
- 11
0
votes
0
answers
82
views
Time Series Hyperparameter Tuning
My question is about the intuition for hyperparameter tuning of time series.
In other models, like Linear or Logistic Regression there is labeled data and according to accuracy or precision, the ...
- 73
0
votes
0
answers
16
views
Validation accuracy have an almost good accuracy but loss function is high too
Based on my project, I find a little problem there with the statement like this.
I want to make model with neural networks for text dataset. Then I use Pad Sequence for my text and Array for the ...
- 13
0
votes
0
answers
20
views
How can i adapt accuracy metric for multiclass classification?
I have a problem which is multiclass e.g. That is 4 classes. I would like a custom metric to assess the model where only if class 3 is predicted as class 2 and class 2 is predicted as class 3 (i.e. ...
- 466
1
vote
0
answers
11
views
How to make an DL model predict Correctly [closed]
So I trained a DL algorithm using Keras for Human Action Recognition. The model has an accuracy of about 85 percent and a loss of 0.3 something. The problem is that the model did not predict well on ...
- 11
0
votes
0
answers
17
views
After finalizing the hyper-parameters and model using a val set, is it recommended to train the final model using both train and validation sets?
Assume I have all the 3 pre-standardized splits of a dataset (train, val, set), with groundtruth for all 3. Usually many datasets are of this form.
Once I have finalized the best configuration of ...
- 131
0
votes
1
answer
252
views
0
votes
0
answers
34
views
Model accuracy and validation accuracy stuck at constant value for denoising autoencoder model
I am building a denoising autoencoder (DAE) to denoise respiratory signals. I pass through the model both noisy and clean versions of the signal (in frame sizes as multiples of 1024).
I've set up my ...
- 13
0
votes
1
answer
104
views
Compare model accuracy when training with imbalanced and balanced data
So I was recently doing a data science project which is a multi class classification.
The project can be found https://www.kaggle.com/c/otto-group-product-classification-challenge.
The dataset is an ...
0
votes
0
answers
110
views
Improve model accuracy in multi-classification problem
I use a MLP to classify three different classes A, B, C. The loss function I use is categorical cross entropy and the optimiser ...
0
votes
1
answer
23
views
Advantages to combining similarly-named columns for supervised ML?
Is there any benefit to combining similarly named columns either for an improvement in accuracy or for speeding up training/prediction in case of logistic regression, random forest or neural network ...
- 11
1
vote
2
answers
83
views
Understanding Sklearns learning_curve
I have been using sklearns learning_curve , and there are a few questions I have that are not answered by the documentation(see also here and here), as well as questions that are raised by the ...
- 161