Questions tagged [accuracy]
The accuracy tag has no usage guidance.
176
questions
0
votes
0answers
4 views
How to calculate performance and change train and test data size in MATLAB NNC Toolbox?
I used MATLAB Application toolbox for clustering but I don't know how to :
Calculate performance and Error rate
How to change test and train data size
Script:
...
1
vote
2answers
30 views
Accuracy of KFold Cross Validation for Neural Network
I have a neural network that Im evaluating using 10 -Fold cross validation. The validation accuracy for a fold changes alot during training in the range of -+10%
So for example the validation ...
0
votes
2answers
43 views
Classification accuracy based on top 3 most likely classifications
My goal is to recommend jobs to job seekers based on their skill set.
Currently I'm using an SVM for this, which is outputting one prediction, e.g. "software engineer at Microsoft". However, consider ...
1
vote
2answers
37 views
Classification accuracy of a Random Multi-label Classifier
What is the exact accuracy of a random classifier which has n labels (say 1000) where k labels (say 50) are true?
Can I say the accuracy of a random classifier has an upper bound of k/n?
-Edit-
I ...
0
votes
0answers
47 views
Prediction error without having a true value
Quick summary about the problem: we are trying to deploy our regression model, where the clients require "individual prediction error". Since we're predicting something unknown in advance, we can't ...
0
votes
0answers
24 views
Variability in CNN test results
I'm trying to do some time series analysis on 1-minute forex data using a CNN. I'm new to deep learning and just getting started in building a model. So this is probably a very basic question, but I'...
0
votes
2answers
41 views
K fold cross validation reduces accuracy
I am working on a machine learning classifier and when I arrive at the moment of dividing my data into training set and test set Iwant to confron two different approches. In one approch I just split ...
1
vote
2answers
23 views
Adding extra variables to XGboost model is worsening the train and test accuracy
I am fitting a multi class model using Xgboost. I am getting an accuracy of 96% on Train and 95% on test. I am using the 80-20 train/test split.
However, when I am adding two new features , the ...
0
votes
1answer
18 views
Can an R^2, or coefficient of determination be used on non-linear data?
I have used the $R^2$ metric to determine how well my neural network performs a non-linear regression. And it seems to work. The plots look almost identical, and I get an $R^2$ value of 0.93... it ...
0
votes
1answer
54 views
Building an efficient feature vector
I am building a classifier for malware analysis, which predicts if I have a malware by looking at the intructions of an assembly code, such as push, mov,... and predicting the optimization method. ...
0
votes
1answer
28 views
KNN scoring low compared to Logistic regression in MNIST challenge
KNN gives me a score of 0.76100 while it shows 94% accuracy for my training data (splitted with test_size =0.3) in my jupyter notebook while logistic regression gives me a score of 0.91485 with an ...
0
votes
1answer
28 views
The loss and accuracy of this LSTM both drop to nearly 0 at the same epoch
I'm trying to train an LSTM to predict the the Nth token using the N-1 tokens preceding it
For each One-Hot encoded token, I ...
1
vote
4answers
134 views
Confusion matrix - determine the values of FP FN TP and TN
After running my code ,I get the values of accuracy, precision and recall and I want t determine the values of FP FN TP and TN from these metrics. I tried to calculate it using the formula of each ...
0
votes
0answers
18 views
How could a considerable increase in loss leads to an improvement in accuracy?
I'm experimenting with NLP and at the moment, I'm trying to come up with a translator model for converting English sentences to French counterparts. I'm using this dataset (not that it's relevant):
...
0
votes
0answers
7 views
Cross_validation is decreasing accuracy?
I have certain dataset to train a model. The dataset is not very small in size. First, I split the dataset into training and validation data using traintestsplit (80-20), train the model on training ...
1
vote
0answers
21 views
Increase accuracy of occupancy prediction?
I have a project that's aimed to predict the amount of occupants at my local gym given the date and weather.
Here's my Kaggle kernel
I have two datasets, occupants on a given hour and weather on a ...
0
votes
0answers
18 views
How to avoid different accuracies when training with subsets?
when trying to train a CNN with randomly selected small subsets (each same size) of the training data set, I get different results in accuracy (the accuracy varies from 0.75 to 0.85).
I determine the ...
1
vote
0answers
29 views
Pytorch testing/validation accuracy over 100%
So I was training my CNN for some hours when it reached 99% accuracy (which was a little bit too good, I thought). But then it didn´t stop and it went higher than 100%. So I thought, that must be ...
0
votes
1answer
132 views
Difference of sklearns accuracy_score() to the commonly accepted Accuracy metric
I am trying to evaluate the accuracy of a multiclass classification setting and I'm wondering why the sklearn implementation of the accuracy score deviates from the commenly agreed on accuracy score: $...
2
votes
1answer
131 views
sklearn.accuracy_score(y_test, y_predict) vs np.mean(y_predict == y_test)
What is the difference between these two methods for finding model accuracy?
I have used both methods in python3 and i normally get identical results. However in few cases i get completely different ...
3
votes
1answer
59 views
How to creat a plot for the accuracy of a model
Iam pretty new to the whole topic so please dont be harsh. I know these may be simple questions but everybody has to start somewhere ^^
So I created (or more copied) my first little Model which ...
0
votes
0answers
66 views
Why does degradation occur in deep neural networks?
It has been shown that "plain" neural networks tend to have an increased amount training error, and accompanied test error, as more layers are added. I am not quite certain as to why this occurs. In ...
0
votes
1answer
31 views
CNN for subsets of a dataset - how to tune hyperparameters
I have a dataset and would like to train CNNs on subsets of different size of the dataset.
I already have a CNN, which classifies very well if I use the entire dataset.
Now the question arises if I ...
0
votes
1answer
17 views
Interpreting a curve val_loss and loss in keras after training a model
I am having trouble understanding the curve val_loss and loss in keras after training my model.
Can anyone help me understand ...
0
votes
1answer
15 views
Sanity check: low PPV but high AUC scores?
I have two algorithms running on a piece of data, both of which perform differently.
One of them (call it A) consistently gets a positive predictive value of about 0.75-0.78. Looking at the AUC of ...
1
vote
0answers
20 views
Binary training result in chainer
I am training simple Chainer based CNN to recognise MNIST samples. To each sample I add poissonian noise. For the test purpose I have always the same random seed. I restart training resetting the ...
2
votes
0answers
173 views
Target mean encoding worse than ordinal encoding with GBDT ( XGBoost, CatBoost )
I have a dataset of 23k rows of an unbalanced dataset 85/15 ratio, 10 variables ( 9 of which are categorical ) , i'm using CatBoost and XGBoost for a binary classification.
I applied cv (5 iteration ...
2
votes
1answer
23 views
How to get the number of steps until a certain accuracy in keras?
I want to see how many steps does it take for my model to reach a certain accuracy.Say 90 percent on cifar10.How can I get this info from the keras model ?
EDIT:
accuracy in each epoch is accessible ...
1
vote
0answers
42 views
30% accuracy for training set, 80% for test set with a 0.3 split
I have a time series dataset on which I am training. For some reason, the training accuracy is 30% while the test accuracy is about 88% after about 10 epochs. Is this at all normal?
I should point ...
0
votes
0answers
43 views
test accuracy with tensorflow==2.0.0-beta1 vs TensorFlow version: 2.0.0-alpha0
I am trying run same piece of code on both Tensorflow==2.0.0-beta1 and TensorFlow version: 2.0.0-alpha0
In TensorFlow version: 2.0.0-alpha0 I am getting Test accuracy: 85.08% but on Tensorflow==2.0.0-...
0
votes
1answer
31 views
Unable to understand the usage of labels argument in sklearn.metrics.f1_score
I am trying to model a dataset with RandomForest Classifier.
My dataset has 3 classes viz. A, B, C. 'A' is the negative class ...
0
votes
0answers
28 views
How to gauge overfit with MLPClassifier and cross_val_score?
I'm learning sklearn.
When using MLPClassifier.fit() and MLPClassifier.predict() I would ...
0
votes
1answer
35 views
Random forest with zero precision for unbalanced test data
Apologies if this is a basic question.
I have a very unbalanced dataset in which the records are labelled by one of two classes, class1 (negative class) and class2 (positive class):
class 1: 1.5 ...
0
votes
0answers
9 views
Is veracity in big data theory a good thing to use big data techniques in organization?
I am new to big data field. I know the basic 3V's of it, but I am not understanding the newly added one, which is Veracity. Is it a good thing, which led to taking great analysis from inaccurate data, ...
1
vote
2answers
57 views
AUC ROC Threshold Setting in heavy imbalance
I am doing binary logistic regression on a dataset with very heavy class imbalance. Class 1 is only 1% of data. When I train logistic regressor without class weights I get ROC AUC Score of 0.6269. ...
0
votes
0answers
19 views
Cost decreases, but accuracy doesn't
It looks like the accuracy is stuck somewhere, I am not sure where and which part is still wrong. Feel free to share your thoughts with me, thanks a lot! :)
...
0
votes
0answers
5 views
Does accuracy scale logarthmic with Learningrates?
This is really general, but:
How influential is a relatively small change in learning rate for any given algorithm?
influences a change from 0.1 to 0.11 in similar magnitude as 0.0001 to 0.1001?
I ...
0
votes
0answers
50 views
How to describe accuracy/error without ground truth?
I am using machine learning regression models to predict motor scores among a population with spinal cord injury using features derived from their actual movements. Although the clinical measure we ...
0
votes
0answers
24 views
The effect of removing pooling layers in the model's accuracy
I know that removing pooling layers will lead to an increase in dimensionality and subsequently, make the training to be more time-consuming. But I'm wondering if ...
0
votes
0answers
11 views
Looking at the graphs, what can I do to increase the accuracy of my neural network
I am trying to solve a regression problem using Keras. The data is time series based and I am using one LSTM layer and two Dense layers.
Following code shows the basic model setup
...
1
vote
1answer
238 views
How to know if a model is overfitting or underfitting by looking at graph
Just recently got my hands on tensorboard, but can you tell me what features should I look for in the graph (Accuracy and Validation Accuracy)
And please do enlighten me about the concept of ...
0
votes
0answers
43 views
Effect Size in comparison of overall accuracy from Random Forest
I would like to compare two overall accuracy statistics (of a Random Forest classifier).
My Data:
two samples with each containing 25 features and one categorical class variables (9 different ...
3
votes
4answers
387 views
Why is my test data accuracy higher than my training data?
I'm using four years of data, training on the first 3 and testing on the fourth. Using LSTM w/ Keras. My test data set (which has no overlap at all with the training) is consistently performing better ...
0
votes
0answers
35 views
MAPE as an accuracy measure
I want to run and compare time-series forecast methods.
Mean Absolute Squared Error (MAPE) is considered one of the strongest metrics for accuracy.
My question is the following:
If you do $1-MAPE$ ...
0
votes
0answers
11 views
Manual setting of target variable based on features' minimum values: f1 score = 1
I am building a classifier for user engagement in my website. Basically, since there are no "proxy" for engagement, i.e. there is no pre-defined target variable, I came up with minimum thresholds ...
0
votes
0answers
189 views
How to improve accuracy of a binary classification in neural network
I tried to program a binary classifier to predict whether a customer belongs to one class or another.
I have over 200k feature vectors, consisting of numeric 12 features each and assume that should ...
2
votes
1answer
191 views
Multiclass classification on imbalanced dataset : Accuracy or micro F1 or macro F1
I have a multiclass classification problem. Further, an instance can be assigned to exactly one class. My dataset is highly imbalanced. I know that accuracy is not a good metric to use in this case ...
1
vote
0answers
31 views
Different accuracy values using the same saved model in tensorflow
I have trained a model in Tensorflow (for some signal classification problem, using mostly convolutional layers, no RNNs), saved It using the callback checkpoints. When I'm testing the said model on a ...
1
vote
2answers
683 views
Loss value going down while accuracy remains constant?
While I am training, it seems like my loss is going down, but my accuracy remains constant throughout training. It always seems to go towards 0.0023 no matter how I tweak my network, input data length,...
0
votes
2answers
613 views
Smaller test data set than training data set in machine learning
I would like to train different machine learning algorithms (SVM, Random Forest, CNN etc.) for the same data set (e.g. MNIST) und then compare their accuracies.
The goal would be to find out from ...
