Questions tagged [roc]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
26 views

How does ROC work with SVM?

Could someone please explain how ROC works with SVM? Specifically i'm using RocCurveDisplay.from_predictions(y_test, y_pred, ax=ax[1]) which works fine. Since the ...
lemintare's user avatar
0 votes
0 answers
25 views

ROC Curve - How to deal with overflow?

I'm trying to calculate the ROC curve and the AUC of a binary logistic regression from scratch, without using third party methods like sklearn.metrics.roc_curve, to ...
Matteo Campagnoli's user avatar
0 votes
1 answer
35 views

ROC curve manual calculation vs. pROC package R

I want do recreate ROC curve manually on my dataset and compare it to roc function from pROC package in R. I'm using dataset on customer churn telco.csv from Kaggle....
Nikola's user avatar
  • 1
4 votes
1 answer
307 views

Does a random classifier have a diagonal ROC (received operator characteristic) curve even when the data is biased toward negatives?

About 9% of the US population have a diabetes diagnosis. So a binary random classifier that just guesses 50% positive and 50% negative would likely be incorrect when it guesses positive (leading to ...
joseville's user avatar
  • 143
1 vote
0 answers
74 views

Macro-average ROC curve not looking right

I am performing a 10-fold Cross-validation on imbalance datasets with small n examples and large p attributes. I am plotting ROC curves by merging predicted probabilities obtained by testing on each k ...
Edoardo Taccaliti's user avatar
0 votes
0 answers
43 views

ROC curve for multiclassification - results sound not correct

I'm working on a multiclassification task using LSTM algorithm, i generated my roc curve plots but they give scores like 1 , 0.99, 0.97 however i have an accuracy of 0.97, Precision 0.65, Sensitivity/...
biihu's user avatar
  • 11
0 votes
1 answer
73 views

calculating tpr and fpr

While calculating the tpr and fpr, can I give both positive class probability or the actual predictions? it give different scores for me, please help me out
CK23's user avatar
  • 68
1 vote
0 answers
122 views

What is ROC curve based on for SVM?

I was studying about the ROC curves for Logistic regression. There is a threshold in this method that determines the classification. By changing this threshold we get different confusion matrices and ...
Mina's user avatar
  • 111
3 votes
1 answer
2k views

Is it sensible to use the ROC curve with an KNN model? And if so why?

I am a beginner doing my first ML project. I am doing a binary supervised classification on an unbalanced dataset and want to use the ROC curve as a performance metric of my models. I am using ...
Ludger's user avatar
  • 33
0 votes
2 answers
435 views

Does it make sense to build a ROC curve for Naive Bayes classification?

These past days, in college, we have been learning about NaiveBayes. Since it's a classification algorithm, I was wondering if I could evaluate NaiveBayes models the same way (using the same metrics) ...
ilved17's user avatar
  • 31
3 votes
1 answer
84 views

What does precision-recall curve and ROC curve tell us abouth threshold invariance

Consider a binary classification problem. Intuitively, a value for the area under the curve (for both curves) very close to 1, shows that the curve is almost L-shaped. Thus, this means that the value ...
liakoyras's user avatar
  • 636
1 vote
3 answers
180 views

ROC Curve for model validation

Is there a general approach that the ROC curve can be used for to validate a model? My understanding is that we can use it to compare different threshold values to determine the best, or even see how ...
user143064's user avatar
2 votes
0 answers
80 views

Smallest possible difference between AUC of two ranker [closed]

If there are 10 positive examples, and 90 negative examples in the test set, what is the smallest possible difference in AUC, between two rankers giving different AUC?
wrek's user avatar
  • 123
2 votes
1 answer
190 views

What makes an ROC curve a curve and why do the values change?

I have a problem. I am currently looking at a classifier and I would like to examine this using an ROC curve as a metric. However, questions have arisen to which I can not find an answer. A ROC curve ...
Test's user avatar
  • 89
1 vote
3 answers
2k views

Inverted ROC curve

I am using tidymodels package in R. Running random forest to classify three classes. There are about 8000 samples in total and 130 features. This is how the ROC curves look like. The predictions for ...
mindlessgreen's user avatar
-1 votes
2 answers
1k views

Why I am having trouble plotting the AUC?

I am trying to plot the roc_auc curve however I am not getting any results. Any explanation here? Are there any problems with the number of data? Here is my example : ...
minattosama's user avatar
2 votes
1 answer
52 views

How do I modify a Logistic Regression to target a specific point on the ROC curve?

From a conceptual standpoint I understand the trade off involved with the ROC curve. You can increase the accuracy of true positive predictions but you will be taking on more false positives and vise ...
Mike's user avatar
  • 21
0 votes
0 answers
326 views

How do I compute the Weighted average ROC Curve?

So i have a multiclass problem and successfully computed the micro and macro average curves, how do I calculate the weighted value for each TPR and FPR?
Marco Ramos's user avatar
1 vote
0 answers
359 views

How to draw each ROC curve of an SVM model with cross validation

I would like to make a graph like the following in python: That is, one curve for each fold. I have the following code where I use an SVM model to classify some data ...
PicaR's user avatar
  • 314
0 votes
2 answers
422 views

Should I be using y_pred or y_pred_proba for binary Classification?

I have a binary classification problem and i want to plot ROC/AUC curve, should I use ypred or ypred_proba
As13's user avatar
  • 77
0 votes
1 answer
488 views

ROC-AUC Imbalanced Data Score Interpretation

I have a binary response variable (label) šµ in a dataset with around 50,000 observations. The training set is somewhat imbalanced with, šµš‘–=1 making up about 33% of the observation's and šµš‘–=0 ...
data wannabe's user avatar
1 vote
1 answer
209 views

How to plot one graph of ROC curve for 4 separate ML model located in different python notebooks

if we have 4 different notebooks for different ML model results .. and we have to plot one ROC curve graph which shows the ROc of all 4 models. how can we do this this is my code in every notebook to ...
user12's user avatar
  • 171
3 votes
1 answer
337 views

Interpreting ROC curves across k-fold cross-validation

I have used a MARS model (multivariate adaptive regression splines) and I have used k fold cross validation for the evaluation of the model, obtaining the following graph: How would be the ...
PicaR's user avatar
  • 314
0 votes
0 answers
38 views

Can you get a very good AUC-ROC score despite predicting all rows to have the same probability?

On the test set of a binary classification problem, the p25, p50 and p75 of the predictions are very close to each other (e.g. 0.123). Is it possible that my model can achieve a high AUC-ROC (e.g. 0....
HK Tong's user avatar
1 vote
0 answers
16 views

My data can be approximated with Normal mixture. How can I find the reasons and explain this behaviour?

I use DeLonge method to compare two ROC AUCS. The result of it is Z-score. Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
Arzental's user avatar
0 votes
2 answers
226 views

What does it mean when roc curves intersect at a point?

I am working with a data set and I have obtained the following roc curve: As you can see, black and Asian ethnicity cross at one point (green and purple lines). Does this have any significance? Could ...
PicaR's user avatar
  • 314
1 vote
0 answers
17 views

How to ensamble different ranking models?

I have trained two different models, which give a score to each data point. The score of the models it is not necessarily comparable. The score is used to give a ranking, and the performance is ...
Diego Palacios's user avatar
1 vote
1 answer
617 views

Improving roc auc score when accuracy is good

I have got a binary classification problem with large dataset of dimensions (1155918, 55) Also dataset is fairly balanced of 67% Class 0 , 33% Class 1. I am getting test accuracy of 73% in test set ...
Shubh's user avatar
  • 198
1 vote
1 answer
14 views

Does thereshold of classifier close to 0 make sense?

I have roc curve with AUC of 0.91. I applied the following function to determine the best threshold: ...
Natili's user avatar
  • 13
3 votes
2 answers
2k views

Uncertainty about shape of ROC curve

I am working on a binary classification and the plotted ROC curves that I am using for evaluation together with AUC, have seemed strange to me. Here is an example. I understand that ROC is a visual ...
lazarea's user avatar
  • 289
0 votes
1 answer
127 views

What happens to auc when true positive rate grows

How does change in true positive rate affects AUC? Does increase of TPR lead to increase of AUC as well?
Ir8_mind's user avatar
  • 183
2 votes
3 answers
97 views

Is roc auc graph better than roc auc score? If yes why?

This was asked in viva of my ML course. I answered yes but could not precisely explain why. By 'better' I mean whether geometric interpretation gives more information than just the numeric score.
Rohit's user avatar
  • 23
0 votes
0 answers
383 views

Use one-vs-rest ROC AUC to get threshold for each class

I have a Neural Network with 4 classes (completely balanced) where the recall is the following for each class (the class with the highest score from the network is chosen) $RE(C1) = 0.611650 $ $RE(C2) ...
CutePoison's user avatar
0 votes
1 answer
1k views

roc_auc_score from sk-learn gives error when test label vector with classes has only a subset of the whole set

I have an imbalanced dataset. Does it make sense to compute the roc-auc for the classifier I created in a holdout set? Here's very artificial MWE: ...
An old man in the sea.'s user avatar
1 vote
1 answer
285 views

Identify which is the best point(s) for (ROC) curve(s)

This is an theorical question, so, I am looking for the point in a ROC Curve. And I got the idea, that different curves has different best point. So, I try to identify those points. For the yellow ...
RMN's user avatar
  • 11
2 votes
1 answer
1k views

Identify optimal thresholds for one-vs-one/one-vs-rest ROC-curve for multiclass classification

Say I have a multiclass classification problem with N classes. I have trained a classifier on a training set, I use a validation set and a One-vs-rest ROC-curve to ...
CutePoison's user avatar
1 vote
2 answers
155 views

If ROC is used to find a threshold, but AUC is threshold invariant, why use AUC?

Say I have a binary classifier. I calculate ROC to select an ideal threshold of say, 0.6. Then, I look at the AUC. But wait! If AUC doesn't change by selecting an 0.6 threshold, then what makes AUC ...
Monica Heddneck's user avatar
0 votes
0 answers
17 views

How can I Determine a Treshold According to the Precision and Recall?

I am gettin these precision and recall values from my classifier and I want to determine a treshold for the test data. How can I determine that treshold? Is these values enough or something else is ...
TarabydaVllasıCafcaflıAtArabsı's user avatar
0 votes
1 answer
231 views

Does it make sense to repeat calculating AUC in logistic regression?

I have a question regarding logistic regression models and testing its skill. I am not quite sure if I understand correctly how the ROC Curve is established. When calculating the ROC curve, is a train ...
DataVader's user avatar
1 vote
0 answers
378 views

ROC and AUC curve for CNN multi-class classification problem

I have produced a convolutional neural network to classify images (malware images) into different classes/families. I have managed to produce a confusion matrix and classification report. My ...
Jack's user avatar
  • 155
1 vote
1 answer
369 views

Overall AUC higher than all "stratified" AUCs

For one of my binary classification models, I have observed this (Simpson's Rule-esque) paradox. The AUC on the test set as a whole is 0.8. Gender is one of the model's features. So I decided to ...
Rohan Kadakia's user avatar
0 votes
1 answer
1k views

Implementing the Trapezoid rule without the formula for the curve

I know that if I have some function f(x) that describes a curve, I can approximate the area under the curve using the trapezoid rule as follows: ...
David Stein's user avatar
6 votes
3 answers
3k views

Can Micro-Average Roc Auc Score be larger than Class Roc Auc Scores

I'm working with an imbalanced data set. There are 11567 negative and 3737 positive samples in train data. There are 2892 negative and 935 positive samples in validation data. It is a binary ...
Angerato's user avatar
3 votes
1 answer
293 views

At what stage are ROC curves used when building machine learning model?

When developing a machine learning model, at what stage are ROC curve with AUC used? Typically I have three data sets train - ...
erotavlas's user avatar
  • 130
1 vote
0 answers
393 views

Best practice to select precision vs. recall threshold for a binary classifier

I have a logistic regression model in Scikit-Learn doing a binary classification. Looking at the Roc curve for the classifier I can see that it performs really well: The AUC score is 0.99 which is ...
Sandy Lee's user avatar
  • 247
3 votes
2 answers
1k views

Interpreting vertical and horizontal parts of ROC curve

It's not clear to me how I can interpret vertical and horizontal parts of the ROC curve. What important information can I gain from this? This is a text from the book "Human-in-the-Loop Machine ...
Mykola Zotko's user avatar
7 votes
1 answer
5k views

Micro Average vs Macro Average for Class Imbalance

I have a dataset consisting of around 30'000 data points and 3 classes. The classes are imbalanced (around 5'000 in class 1, 10'000 in class 2 and 15'000 in class 3). I'm building a convolutional ...
machinery's user avatar
  • 246
1 vote
1 answer
194 views

Is it possible to get an ROC curve using Relu activation?

Based on my understanding, given that Relu doesn't provide probabilities unlike Softmax, it's not possible to plot an ROC curve. However, is there some way to convert the output from a Relu to ...
sickerin's user avatar
  • 113
1 vote
1 answer
63 views

How would you interpret the following ROC and PRC curves?

How would you interpret the following ROC and PRC curves? For example, I find it weird to understand that the precision actually increases at some point when recall increases as well. Is that even ...
Matt's user avatar
  • 125
4 votes
0 answers
199 views

Fast PR / ROC curves and corespondings AUPR / AUROC

I find myself in a position of calculating numerous PR / ROC curves and their associated area under the PR curves (AUPR) / area under the ROC curve (AUROC). Its is quite easy to perform those ...
Lucas Morin's user avatar
  • 2,196