Questions tagged [accuracy]
In data science, accuracy is a measurement used to determine which model is best at describing the underlying patterns of a dataset.
403
questions
1
vote
1
answer
22
views
How to deal with categorical disalignment in test and train in binary classification problems
I have a train and test datasets (600k observations) that have different categories for the same categorical variable.
For example train has the categorical variable Letters having unique categories ...
- 13
0
votes
0
answers
10
views
Finding Accuracy, Recall, Precision, and F1 from Matlab Confusion Matrix
I'm working on a project to find the highest accuracy between KNN and a Decision Tree for Classification using Matlab.
How to calculate the Accuracy, Recall, Precision, and F1 from the output below? ...
- 1
2
votes
1
answer
117
views
Why the test accuracy showing some odd behaviour in comparison to train accuracy?
I am currently training an ANN using Sequential(a class from Keras API within tensorflow), and I am optimizing the model's architecture and came across something I have not seen before.
The graph of ...
- 23
0
votes
0
answers
23
views
How to infer the following graphs for dimensionality reduction?
I'm dealing with a high-dimensional(1600 features, 9500 columns) binary classification problem. My current accuracy, and other metrics are not upto the mark. I am trying different feature selection ...
1
vote
1
answer
220
views
How to calculate the training accuracy of a decision tree?
The hint given was to construct a confusion matrix.
0
votes
0
answers
18
views
Improving Normalizing Flows Accuracy
What are some techniques one might use to improve the accuracy of normalizing flows? I am training a flow in a high-dimensional space but it seems like there's always at least one or two dimensions in ...
- 1
0
votes
0
answers
10
views
warning 'newdata' had X row but variables found have Y rows
Linear Discriminant Analysis (LDA)+logistic regression model
lda_model <- lda(train_labels ~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width, data = train_data)
LDA scores for the training ...
1
vote
0
answers
23
views
Validation Accuracy incorrect when multiple outputs are in the dense layer!
I have a set-up a parallel LSTM architecture with two LSTM layers and one Dense layer producing several outputs which are converted into a probability with a sigmoid function on the dense layer. For ...
- 11
0
votes
1
answer
40
views
Is there a way to focus mainly on high precision when fitting a tree model?
I have a dataset with 95% false and 5% true labels, some 200000 samples overall, I'm fitting a LightGBM model. I mainly need to focus on high precision and have low number of false positives, I don't ...
- 1
0
votes
0
answers
23
views
Is it correct to use t-test on accuracy values of two different classifiers?
I have two datasets (500 data points with 15 variables and a binary output):
The first one includes all variables.
The second one includes all variables, except one is removed.
I want to check the ...
- 1
0
votes
1
answer
28
views
Why did both my precision and recall (hence also F1) improve dramatically but the accuracy lowered?
I split all the audio recordings in my small dataset into short clips, thereby "created" more samples. All of precision, recall and F1 almost doubled (from around 0.35 to around 0.65), but ...
0
votes
1
answer
31
views
Is it possible to detect early if a model is bad?
Let's say we have a model and have just started to fit it, the first epoch out of many. The first epoch shows awful results. Does it make sense to continue training hoping the results will be better ...
- 105
0
votes
0
answers
23
views
Decision making in a binary classification problem
Consider a two-dimensional feature space in which the line $\mathbf{w}.\mathbf{x} + b = 0$, where $ \mathbf{w},\mathbf{x} \in \mathbb{R}^ 2 $ and $b \in \mathbb{R}$, separates linearly separable data ...
-1
votes
2
answers
76
views
Recall and Precision ML models
I use decision trees for a binary classification. To evaluate the model, I use K-fold cross-validation, where k = 10.
When I run the model n times, I get a relatively constant accuracy across all ...
1
vote
1
answer
265
views
How to measure accuracy of GPT model
I am working on a model to build questions automatically from some text
My model will analyse provided article and ask authors questions that can help improving their articles
How can we measure the ...
- 539
-1
votes
1
answer
34
views
Which is the best binary classification model? Train and Test Accuracy are similar
I am building a binary classification model where classes are imbalanced but used SMOTE, I used 4 different models to compare performance and decide which to choose.
They have same train and test ...
- 1
0
votes
1
answer
30
views
What is the highest possible prediction accuracy when I flip some labels at random?
I want to predict MNIST labels in a binary setting using a simple MLP model (0 for digits 0-4 and 1 for 5-9). For the train and test data, I randomly flip 25% of the labels.
Is the maximum achievable ...
-1
votes
1
answer
23
views
How to Data Engineer a dataset to get the best featurres to predict a target class?
In my dataset, I have data of IDs that don't create any meaningful relationship with each other and when I test that dataset on different models I am not getting accuracy more than 40%.
Anyone can ...
0
votes
1
answer
279
views
Torchmetrics Binary Accuracy and Multiclass Accuracy don't match
in my program I have the problem that for a 2-class classification problem my multiclass accuracy and binary class accuracy don't match.
I have generated a very small sample example where you can see ...
- 3
1
vote
1
answer
145
views
Why is accuracy score suddenly becoming 1 on using XGBoost?
I am developing a music classification system based on a kaggle dataset: https://www.kaggle.com/datasets/vatsalmavani/spotify-dataset
I tried using K means classifier to classify the songs into 4 ...
0
votes
0
answers
33
views
How can I improve accuracy of my ensemble (or anywhere in the code where I can increase accuracy)?
I am pretty new to machine learning, so if my code is not good, please bear with me.
...
0
votes
2
answers
164
views
KNN Accuracy training
After I developed my model using KNN I get the following accuracy:
Train Accuracy :: 1
Test Accuracy :: 0.24
What is the accuracy of my model?
1
vote
1
answer
73
views
Which preprocessing is the correct way to forecast time-series data using LSTM?
I just started to study time-series forecasting using RNN.
I have a few months of time series data that was an hour unit.
The data is a kind of percentage value of my little experiment and no other ...
- 113
0
votes
2
answers
192
views
which Keras accuracy metric for multiclass classification
I am training a CNN for multiclass image classification into 4 images , what accuracy metric should i use from Keras. My labels are not one hot encoded as I am trying to predict probability of ...
0
votes
0
answers
21
views
How to refine sampling of points via averaging?
I am working with a generative model which is generating points that are less accurate than I would like. I have strong reason to believe the errors should average out along a particular axis (to give ...
- 1
0
votes
0
answers
114
views
Ordinal logistic regression prediction and accuracy using statsmodels
I am trying to do a ordinal logistic regression analysis using statsmodels. However, the predictions I'm getting are vastly different from that I get when using SciKit-Learn ...
- 101
0
votes
1
answer
64
views
Testing accuracy is higher than training accuracy
My testing accuracy is way higher than my training accuracy. I have used feature selection and split the data into training, validation and test sets.
...
- 3
0
votes
0
answers
34
views
My test dataset accuracy score is 1.0 for Random Forest - is the issue to do with my code or my dataset?
I use these classification algorithms to test whether a URL is phishing or legitimate.
I have tried feature_importance, cross-validation, and hyperparameter tuning to the best of my ability, but I ...
- 1
0
votes
0
answers
15
views
Batch normalization getting Val accuracy of 000e00
I am currently stuck on batch normalization. I have code written out but when I do it it keeps giving me a val_accuracy of 00e00 and stops after 2 epochs (I am wanting to run 20). I am not sure what ...
1
vote
2
answers
86
views
Neural network not learning at all
I am training a MLP on a tabular dataset, the pendigits dataset. Problem is that training loss and accuracy are more or less stable, while validation and test loss and accuracy are completely constant....
- 229
0
votes
2
answers
63
views
I am getting all scores as 100% on my machine learning models. Is it okay to have this kind of result?
I am getting all scores for my ML model as 100% for the Extra Trees Algorithm. I am applying the necessary pre-processing steps (duplication removal, correlations validating, cardinality validation, ...
- 1
0
votes
1
answer
65
views
how do I test if overfitting exists when I use cross_val_score method?
I got the following code form a book on xgboost.
I wonder whether this is a correct way of analyzing cross validation score for overfitting purposes.
mean accuracy is 81 which can be okay. but what if ...
- 31
0
votes
1
answer
131
views
Why is the accuracy on train dataset not always 100% while we use the same dataset to train the model?
Though tree-based ML algorithms give us 100% accuracy on train dataset many times, but why is this not happening every time. I know this results in overfitting but why not 100% accuracy every time on ...
0
votes
1
answer
764
views
How to calculate accuracy of a logistic regression?
A logistic regression involves a linear combination of features to predict the log-odds of a binary, yes/no-style event. That log-odds can then be transformed to a probability. If $\hat L_i$ is the ...
- 3,919
0
votes
0
answers
16
views
Text Classification Model unable to learn
I am trying to build a text classification model. When I train the model it is unable to improve accuracy and at some point accuracy even decreases and loss increases.
I have researched for possible ...
- 1
0
votes
0
answers
119
views
Regularization in CNN change training accuracy but not validation accuracy
I have a question regarding regularization in convolutional neural networks.
So I'm building a CNN for image classification and I've come across something I don't understand.
Without using any ...
0
votes
0
answers
47
views
FinBERT out of the box performance testing
I'm trying to perform an out of the box performance test for FinBERT using the financialphrasebank dataset(sentiment analysis) to get a baseline performance before I start finetuning the model. The ...
- 1
0
votes
1
answer
42
views
Do I need to use always the same "Test" dataset to compare between different models?
I have two datasources A and B, and I want to check how several methods can affect the accuracy of my multi class models:
If I use cross-validation with validate dataset to obtain the best
hyper ...
0
votes
1
answer
56
views
A curve val_loss and loss in keras after training a model
Can anyone help me, is my model overfitting or underfitting?
I want to make sure the model is well done before starting to deploy
Also, I use categorical cross-entropy loss
I have asked before, but I ...
- 1
0
votes
0
answers
36
views
Robustness of an ARIMA Model
I have built an ARIMAX model in python for predicting a time series. After presenting my findings ive been asked what robustness tests i have used. My skillset is more on the python side. I only have ...
- 131
2
votes
4
answers
2k
views
99% accuracy in train and 96% in test is too much overfitting?
I have a binary classification problem, the classes are quite balanced (57%-43%), with a GridSearch with Random Forest Classifier I obtained the best hyperparameters and I applied the model to train ...
- 41
0
votes
1
answer
76
views
I get 100% on my test set using random forest. What is wrong?
I am getting 100% accuracy on my test set when trained using random forest.
Is there something wrong with my model?
Code:
...
- 3
0
votes
0
answers
20
views
How does backpropagation through accuracy work?
I'm using a specific constraint on my predicted logits and adding it to the loss.
In a nutshell, this constraint tries to minimize cross-overlap between the channels of my predictions. I'm using ...
- 13
2
votes
1
answer
3k
views
Accuracy vs Categorical Accuracy
I was running a DNN model that uses ResNet50 for Transfer Learning. While fitting the training data on my model to check the initial trend (would run for more epochs if initial trend seems right), I ...
- 77
2
votes
1
answer
695
views
Validation and training loss of a model are not stable
Below I have a model trained and the loss of both the training dataset (blue) and validation dataset (orange) are shown. From my understanding, the ideal case is that both validation and training loss ...
- 231
0
votes
0
answers
24
views
Accuracy not Increasing in fake news classifier
I have been trying to build a fake news classifier DL model.
Notebook here
Problem is when i put the same data in a Multinomial model it gives good accuracy.
But ...
- 135
1
vote
1
answer
52
views
What does that mean if the loss looks like this?
I have a problem. I have trained a model. And as you can see, there is a zigzag in the loss. In addition, the validation loss is increasing.
What does this mean if you only look at the training curve? ...
- 89
0
votes
1
answer
68
views
How to Improve MLP ANN accuracy
I am trying to improve the accuracy of my model over the UCI Breast Cancer Dataset. There's 426 records, and it is a binary classification model.
...
- 1
0
votes
1
answer
28
views
Is it possible to train a Support Vector Machine to a specific accuracy?
From my understanding, support vector machines run on the premise of minimizing some error function, usually with the goal of maximizing accuracy overall. However, there are a lot of contexts, ...
- 27
1
vote
1
answer
34
views
Can we train of a binary classifier with "A" to classify "a"?
I have a maybe naive question about the appropriateness of using binary classifications. This is a hypothetical example, so forgive me if it is too coarse.
Let's say I want to train a support vector ...
- 11