Questions tagged [roc]
The roc tag has no usage guidance.
56
questions
0
votes
0
answers
19
views
ROC curve for multiclassification - results sound not correct
I'm working on a multiclassification task using LSTM algorithm, i generated my roc curve plots but they give scores like 1 , 0.99, 0.97 however i have an accuracy of 0.97, Precision 0.65, Sensitivity/...
- 11
0
votes
1
answer
42
views
calculating tpr and fpr
While calculating the tpr and fpr, can I give both positive class probability or the actual predictions? it give different scores for me, please help me out
- 68
1
vote
0
answers
70
views
What is ROC curve based on for SVM?
I was studying about the ROC curves for Logistic regression. There is a threshold in this method that determines the classification. By changing this threshold we get different confusion matrices and ...
- 111
2
votes
1
answer
1k
views
Is it sensible to use the ROC curve with an KNN model? And if so why?
I am a beginner doing my first ML project. I am doing a binary supervised classification on an unbalanced dataset and want to use the ROC curve as a performance metric of my models. I am using ...
- 23
0
votes
2
answers
157
views
Does it make sense to build a ROC curve for Naive Bayes classification?
These past days, in college, we have been learning about NaiveBayes. Since it's a classification algorithm, I was wondering if I could evaluate NaiveBayes models the same way (using the same metrics) ...
- 31
3
votes
1
answer
39
views
What does precision-recall curve and ROC curve tell us abouth threshold invariance
Consider a binary classification problem.
Intuitively, a value for the area under the curve (for both curves) very close to 1, shows that the curve is almost L-shaped.
Thus, this means that the value ...
- 626
1
vote
3
answers
83
views
ROC Curve for model validation
Is there a general approach that the ROC curve can be used for to validate a model? My understanding is that we can use it to compare different threshold values to determine the best, or even see how ...
- 11
0
votes
0
answers
8
views
Low value of area under PR and high value of a area under ROC interpretation?
I am still a rookie. I am training a random forest model and I am getting 0.27 for area under PR and 0.85 for area under ROC. My data was very imbalanced for the negative labels and I did perform a ...
- 101
2
votes
0
answers
72
views
Smallest possible difference between AUC of two ranker [closed]
If there are 10 positive examples, and 90 negative examples in the test set, what is the smallest possible difference in AUC, between two rankers giving different AUC?
- 123
2
votes
1
answer
106
views
What makes an ROC curve a curve and why do the values change?
I have a problem. I am currently looking at a classifier and I would like to examine this using an ROC curve as a metric. However, questions have arisen to which I can not find an answer.
A ROC curve ...
- 89
1
vote
3
answers
781
views
Inverted ROC curve
I am using tidymodels package in R. Running random forest to classify three classes. There are about 8000 samples in total and 130 features. This is how the ROC curves look like.
The predictions for ...
- 111
-1
votes
2
answers
602
views
Why I am having trouble plotting the AUC?
I am trying to plot the roc_auc curve however I am not getting any results. Any explanation here? Are there any problems with the number of data?
Here is my example :
...
- 204
2
votes
1
answer
41
views
How do I modify a Logistic Regression to target a specific point on the ROC curve?
From a conceptual standpoint I understand the trade off involved with the ROC curve. You can increase the accuracy of true positive predictions but you will be taking on more false positives and vise ...
- 21
0
votes
0
answers
284
views
How do I compute the Weighted average ROC Curve?
So i have a multiclass problem and successfully computed the micro and macro average curves, how do I calculate the weighted value for each TPR and FPR?
- 11
1
vote
0
answers
286
views
How to draw each ROC curve of an SVM model with cross validation
I would like to make a graph like the following in python:
That is, one curve for each fold.
I have the following code where I use an SVM model to classify some data
...
- 304
0
votes
2
answers
305
views
Should I be using y_pred or y_pred_proba for binary Classification?
I have a binary classification problem and i want to plot ROC/AUC curve, should I use ypred or ypred_proba
- 47
0
votes
1
answer
302
views
ROC-AUC Imbalanced Data Score Interpretation
I have a binary response variable (label) 𝐵 in a dataset with around 50,000 observations.
The training set is somewhat imbalanced with, 𝐵𝑖=1 making up about 33% of the observation's and 𝐵𝑖=0 ...
- 21
1
vote
1
answer
174
views
How to plot one graph of ROC curve for 4 separate ML model located in different python notebooks
if we have 4 different notebooks for different ML model results .. and we have to plot one ROC curve graph which shows the ROc of all 4 models. how can we do this
this is my code in every notebook to ...
- 171
3
votes
1
answer
268
views
Interpreting ROC curves across k-fold cross-validation
I have used a MARS model (multivariate adaptive regression splines) and I have used k fold cross validation for the evaluation of the model, obtaining the following graph:
How would be the ...
- 304
0
votes
0
answers
33
views
Can you get a very good AUC-ROC score despite predicting all rows to have the same probability?
On the test set of a binary classification problem, the p25, p50 and p75 of the predictions are very close to each other (e.g. 0.123).
Is it possible that my model can achieve a high AUC-ROC (e.g. 0....
- 1
1
vote
0
answers
16
views
My data can be approximated with Normal mixture. How can I find the reasons and explain this behaviour?
I use DeLonge method to compare two ROC AUCS. The result of it is Z-score.
Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
- 11
0
votes
2
answers
157
views
What does it mean when roc curves intersect at a point?
I am working with a data set and I have obtained the following roc curve:
As you can see, black and Asian ethnicity cross at one point (green and purple lines). Does this have any significance?
Could ...
- 304
1
vote
0
answers
16
views
How to ensamble different ranking models?
I have trained two different models, which give a score to each data point. The score of the models it is not necessarily comparable.
The score is used to give a ranking, and the performance is ...
- 111
1
vote
1
answer
335
views
Improving roc auc score when accuracy is good
I have got a binary classification problem with large dataset of dimensions (1155918, 55)
Also dataset is fairly balanced of 67% Class 0 , 33% Class 1.
I am getting test accuracy of 73% in test set ...
- 198
1
vote
1
answer
14
views
Does thereshold of classifier close to 0 make sense?
I have roc curve with AUC of 0.91. I applied the following function to determine the best threshold: ...
- 13
3
votes
2
answers
1k
views
Uncertainty about shape of ROC curve
I am working on a binary classification and the plotted ROC curves that I am using for evaluation together with AUC, have seemed strange to me. Here is an example.
I understand that ROC is a visual ...
- 289
0
votes
1
answer
92
views
What happens to auc when true positive rate grows
How does change in true positive rate affects AUC? Does increase of TPR lead to increase of AUC as well?
- 183
2
votes
3
answers
73
views
Is roc auc graph better than roc auc score? If yes why?
This was asked in viva of my ML course. I answered yes but could not precisely explain why.
By 'better' I mean whether geometric interpretation gives more information than just the numeric score.
- 23
0
votes
0
answers
353
views
Use one-vs-rest ROC AUC to get threshold for each class
I have a Neural Network with 4 classes (completely balanced) where the recall is the following for each class (the class with the highest score from the network is chosen)
$RE(C1) = 0.611650 $
$RE(C2) ...
- 450
0
votes
1
answer
1k
views
roc_auc_score from sk-learn gives error when test label vector with classes has only a subset of the whole set
I have an imbalanced dataset.
Does it make sense to compute the roc-auc for the classifier I created in a holdout set?
Here's very artificial MWE:
...
1
vote
1
answer
153
views
Identify which is the best point(s) for (ROC) curve(s)
This is an theorical question, so, I am looking for the point in a ROC Curve. And I got the idea, that different curves has different best point.
So, I try to identify those points.
For the yellow ...
- 11
2
votes
1
answer
882
views
Identify optimal thresholds for one-vs-one/one-vs-rest ROC-curve for multiclass classification
Say I have a multiclass classification problem with N classes.
I have trained a classifier on a training set, I use a validation set and a One-vs-rest ROC-curve to ...
- 450
1
vote
2
answers
111
views
If ROC is used to find a threshold, but AUC is threshold invariant, why use AUC?
Say I have a binary classifier. I calculate ROC to select an ideal threshold of say, 0.6.
Then, I look at the AUC.
But wait! If AUC doesn't change by selecting an 0.6 threshold, then what makes AUC ...
- 477
0
votes
0
answers
16
views
How can I Determine a Treshold According to the Precision and Recall?
I am gettin these precision and recall values from my classifier and I want to determine a treshold for the test data. How can I determine that treshold? Is these values enough or something else is ...
0
votes
1
answer
150
views
Does it make sense to repeat calculating AUC in logistic regression?
I have a question regarding logistic regression models and testing its skill.
I am not quite sure if I understand correctly how the ROC Curve is established.
When calculating the ROC curve, is a train ...
- 15
1
vote
0
answers
330
views
ROC and AUC curve for CNN multi-class classification problem
I have produced a convolutional neural network to classify images (malware images) into different classes/families. I have managed to produce a confusion matrix and classification report. My ...
- 145
1
vote
1
answer
258
views
Overall AUC higher than all "stratified" AUCs
For one of my binary classification models, I have observed this (Simpson's Rule-esque) paradox. The AUC on the test set as a whole is 0.8.
Gender is one of the model's features. So I decided to ...
- 147
0
votes
1
answer
973
views
Implementing the Trapezoid rule without the formula for the curve
I know that if I have some function f(x) that describes a curve, I can approximate the area under the curve using the trapezoid rule as follows:
...
- 165
6
votes
3
answers
3k
views
Can Micro-Average Roc Auc Score be larger than Class Roc Auc Scores
I'm working with an imbalanced data set. There are 11567 negative and 3737 positive samples in train data. There are 2892 negative and 935 positive samples in validation data. It is a binary ...
- 63
3
votes
1
answer
258
views
At what stage are ROC curves used when building machine learning model?
When developing a machine learning model, at what stage are ROC curve with AUC used?
Typically I have three data sets
train - ...
- 130
1
vote
0
answers
313
views
Best practice to select precision vs. recall threshold for a binary classifier
I have a logistic regression model in Scikit-Learn doing a binary classification.
Looking at the Roc curve for the classifier I can see that it performs really well:
The AUC score is 0.99 which is ...
- 227
3
votes
2
answers
925
views
Interpreting vertical and horizontal parts of ROC curve
It's not clear to me how I can interpret vertical and horizontal parts of the ROC curve. What important information can I gain from this? This is a text from the book "Human-in-the-Loop Machine ...
- 57
6
votes
1
answer
5k
views
Micro Average vs Macro Average for Class Imbalance
I have a dataset consisting of around 30'000 data points and 3 classes. The classes are imbalanced (around 5'000 in class 1, 10'000 in class 2 and 15'000 in class 3). I'm building a convolutional ...
- 236
1
vote
1
answer
180
views
Is it possible to get an ROC curve using Relu activation?
Based on my understanding, given that Relu doesn't provide probabilities unlike Softmax, it's not possible to plot an ROC curve. However, is there some way to convert the output from a Relu to ...
- 113
1
vote
1
answer
59
views
How would you interpret the following ROC and PRC curves?
How would you interpret the following ROC and PRC curves?
For example, I find it weird to understand that the precision actually increases at some point when recall increases as well. Is that even ...
- 125
3
votes
0
answers
71
views
Fast PR / ROC curves and corespondings AUPR / AUROC
I find myself in a position of calculating numerous PR / ROC curves and their associated area under the PR curves (AUPR) / area under the ROC curve (AUROC).
Its is quite easy to perform those ...
- 2,513
0
votes
1
answer
137
views
Correctness of a ROC Curve
I've built a Decision Tree Classifier to practice with the sklearn library. My first task was to shuffle the iris dataset and split it keeping only the last 10 elements for the test. Then, after the ...
- 59
2
votes
0
answers
2k
views
AUC on ROC Curve near 1.0 for Multi-Class CNN but Precision/Recall are not perfect?
I am building a ROC Curve and calculating AUC for multi-class classification on the CIFAR-10 dataset using a CNN. My overall Accuracy is ~ 90% and my precision and recall are as follows:
...
- 159
0
votes
1
answer
700
views
AUC ROC Curve multi class Classification
Here is the part of the code for ROC AUC Curve calculation for multiple classes.
...
- 175
2
votes
1
answer
147
views
What is the difference in plotting ROC curve with probability scores vs binary decisions
As the title reads what is the difference? Plotting the ROC w.r.t probability scores gives the stair cased version.
But in my opinion I find that using binary decision is better because the ROC curve ...
- 21