Questions tagged [roc]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
-1 votes
2 answers
21 views

Why I am having trouble plotting the AUC?

I am trying to plot the roc_auc curve however I am not getting any results. Any explanation here? Are there any problems with the number of data? Here is my example : ...
user avatar
0 votes
0 answers
9 views

Why would an ensemble model perform worse than all individual models? biomod2

I'm using biomod2 in R and my ensemble model performs worse on the evaluation data (drastically lower ROC, 0.835) than any of the individual models (ROC ranges 0.89-0.97). What could be causing this? ...
user avatar
0 votes
0 answers
15 views

Why is ROC-AUC usually shown in GNN papers

In various graph neural network (GNNs) papers, the ROC-AUC metric is usually shown alone without considering F1 or Accuracy. Is there a reason for that? What does it say about two models 1 and 2 with ...
user avatar
0 votes
0 answers
18 views

How to draw a ROC curve for each Fold in cross validation in R

I am evaluating my model using K fold cross validation and I would like to draw a ROC curve for each of the folds and show them ALL TOGETHER. I'm using the R programming language and I'm going to ...
user avatar
  • 302
0 votes
0 answers
19 views

How to draw the ROC curve of a classifier evaluated with cross validation

I was wondering if anyone knows without with the R programming language, given a classifier evaluated by k fold cross validation, I can draw each of the ROC curves that are generated in each fold of ...
user avatar
  • 302
2 votes
1 answer
30 views

How do I modify a Logistic Regression to target a specific point on the ROC curve?

From a conceptual standpoint I understand the trade off involved with the ROC curve. You can increase the accuracy of true positive predictions but you will be taking on more false positives and vise ...
user avatar
  • 21
0 votes
0 answers
56 views

How do I compute the Weighted average ROC Curve?

So i have a multiclass problem and successfully computed the micro and macro average curves, how do I calculate the weighted value for each TPR and FPR?
user avatar
1 vote
0 answers
93 views

How to draw each ROC curve of an SVM model with cross validation

I would like to make a graph like the following in python: That is, one curve for each fold. I have the following code where I use an SVM model to classify some data ...
user avatar
  • 302
0 votes
2 answers
70 views

Should I be using y_pred or y_pred_proba for binary Classification?

I have a binary classification problem and i want to plot ROC/AUC curve, should I use ypred or ypred_proba
user avatar
  • 27
0 votes
0 answers
9 views

How are ROC curves constructed? [duplicate]

I would like to understand how to build a ROC curve of a model. For example, if we would like to draw it by hand, what steps should we do? Thank you.
user avatar
  • 302
0 votes
1 answer
37 views

ROC-AUC Imbalanced Data Score Interpretation

I have a binary response variable (label) 𝐵 in a dataset with around 50,000 observations. The training set is somewhat imbalanced with, 𝐵𝑖=1 making up about 33% of the observation's and 𝐵𝑖=0 ...
user avatar
1 vote
1 answer
34 views

How to plot one graph of ROC curve for 4 separate ML model located in different python notebooks

if we have 4 different notebooks for different ML model results .. and we have to plot one ROC curve graph which shows the ROc of all 4 models. how can we do this this is my code in every notebook to ...
user avatar
  • 175
3 votes
1 answer
139 views

Interpreting ROC curves across k-fold cross-validation

I have used a MARS model (multivariate adaptive regression splines) and I have used k fold cross validation for the evaluation of the model, obtaining the following graph: How would be the ...
user avatar
  • 302
0 votes
0 answers
15 views

Why has only sense to show the Separation criteria in the ROC plot?

I am dealing with the issue of fairness in machine learning models. One of the group fairness criteria is separation. I have read that it only makes sense to show the separation criterion using ROC ...
user avatar
  • 302
0 votes
0 answers
19 views

Can you get a very good AUC-ROC score despite predicting all rows to have the same probability?

On the test set of a binary classification problem, the p25, p50 and p75 of the predictions are very close to each other (e.g. 0.123). Is it possible that my model can achieve a high AUC-ROC (e.g. 0....
user avatar
0 votes
0 answers
9 views

How to observe dependencies using ROC curves?

I am dealing with a database in which I am recording the scores that each race has to obtain a credit. I have made the following graphs: [enter image description here]2 Where you can see the relative ...
user avatar
  • 302
1 vote
0 answers
15 views

My data can be approximated with Normal mixture. How can I find the reasons and explain this behaviour?

I use DeLonge method to compare two ROC AUCS. The result of it is Z-score. Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
user avatar
0 votes
2 answers
70 views

What does it mean when roc curves intersect at a point?

I am working with a data set and I have obtained the following roc curve: As you can see, black and Asian ethnicity cross at one point (green and purple lines). Does this have any significance? Could ...
user avatar
  • 302
1 vote
0 answers
12 views

How to ensamble different ranking models?

I have trained two different models, which give a score to each data point. The score of the models it is not necessarily comparable. The score is used to give a ranking, and the performance is ...
user avatar
1 vote
1 answer
66 views

Improving roc auc score when accuracy is good

I have got a binary classification problem with large dataset of dimensions (1155918, 55) Also dataset is fairly balanced of 67% Class 0 , 33% Class 1. I am getting test accuracy of 73% in test set ...
user avatar
  • 198
1 vote
1 answer
12 views

Does thereshold of classifier close to 0 make sense?

I have roc curve with AUC of 0.91. I applied the following function to determine the best threshold: ...
user avatar
  • 13
3 votes
2 answers
571 views

Uncertainty about shape of ROC curve

I am working on a binary classification and the plotted ROC curves that I am using for evaluation together with AUC, have seemed strange to me. Here is an example. I understand that ROC is a visual ...
user avatar
  • 269
0 votes
1 answer
43 views

What happens to auc when true positive rate grows

How does change in true positive rate affects AUC? Does increase of TPR lead to increase of AUC as well?
user avatar
  • 153
0 votes
0 answers
43 views

Is it possible to use roc auc metric in uplift modeling (class transformatio approach)

I do not understand why in uplift modeling (Class Transformation approach) not used ROC AUC score for changed target Z. I have a problem with a task where I tried to use this approach, but ROC AUC ...
user avatar
2 votes
3 answers
55 views

Is roc auc graph better than roc auc score? If yes why?

This was asked in viva of my ML course. I answered yes but could not precisely explain why. By 'better' I mean whether geometric interpretation gives more information than just the numeric score.
user avatar
  • 23
0 votes
0 answers
131 views

Use one-vs-rest ROC AUC to get threshold for each class

I have a Neural Network with 4 classes (completely balanced) where the recall is the following for each class (the class with the highest score from the network is chosen) $RE(C1) = 0.611650 $ $RE(C2) ...
user avatar
0 votes
1 answer
481 views

roc_auc_score from sk-learn gives error when test label vector with classes has only a subset of the whole set

I have an imbalanced dataset. Does it make sense to compute the roc-auc for the classifier I created in a holdout set? Here's very artificial MWE: ...
user avatar
1 vote
1 answer
36 views

Identify which is the best point(s) for (ROC) curve(s)

This is an theorical question, so, I am looking for the point in a ROC Curve. And I got the idea, that different curves has different best point. So, I try to identify those points. For the yellow ...
user avatar
  • 11
0 votes
0 answers
179 views

How can I compute the AUC by using Gaussian Mixture Model?

By using this code, can I compute the AUC: ...
user avatar
  • 21
2 votes
1 answer
360 views

Identify optimal thresholds for one-vs-one/one-vs-rest ROC-curve for multiclass classification

Say I have a multiclass classification problem with N classes. I have trained a classifier on a training set, I use a validation set and a One-vs-rest ROC-curve to ...
user avatar
1 vote
2 answers
61 views

If ROC is used to find a threshold, but AUC is threshold invariant, why use AUC?

Say I have a binary classifier. I calculate ROC to select an ideal threshold of say, 0.6. Then, I look at the AUC. But wait! If AUC doesn't change by selecting an 0.6 threshold, then what makes AUC ...
user avatar
0 votes
0 answers
15 views

How can I Determine a Treshold According to the Precision and Recall?

I am gettin these precision and recall values from my classifier and I want to determine a treshold for the test data. How can I determine that treshold? Is these values enough or something else is ...
user avatar
0 votes
1 answer
81 views

Does it make sense to repeat calculating AUC in logistic regression?

I have a question regarding logistic regression models and testing its skill. I am not quite sure if I understand correctly how the ROC Curve is established. When calculating the ROC curve, is a train ...
user avatar
1 vote
0 answers
194 views

ROC and AUC curve for CNN multi-class classification problem

I have produced a convolutional neural network to classify images (malware images) into different classes/families. I have managed to produce a confusion matrix and classification report. My ...
user avatar
  • 125
1 vote
1 answer
123 views

Overall AUC higher than all "stratified" AUCs

For one of my binary classification models, I have observed this (Simpson's Rule-esque) paradox. The AUC on the test set as a whole is 0.8. Gender is one of the model's features. So I decided to ...
user avatar
0 votes
1 answer
134 views

Logistic Regression optimal threshold is a negative value

I run the code below: ...
user avatar
0 votes
1 answer
386 views

Implementing the Trapezoid rule without the formula for the curve

I know that if I have some function f(x) that describes a curve, I can approximate the area under the curve using the trapezoid rule as follows: ...
user avatar
6 votes
3 answers
1k views

Can Micro-Average Roc Auc Score be larger than Class Roc Auc Scores

I'm working with an imbalanced data set. There are 11567 negative and 3737 positive samples in train data. There are 2892 negative and 935 positive samples in validation data. It is a binary ...
user avatar
3 votes
1 answer
192 views

At what stage are ROC curves used when building machine learning model?

When developing a machine learning model, at what stage are ROC curve with AUC used? Typically I have three data sets train - ...
user avatar
  • 130
1 vote
0 answers
166 views

Best practice to select precision vs. recall threshold for a binary classifier

I have a logistic regression model in Scikit-Learn doing a binary classification. Looking at the Roc curve for the classifier I can see that it performs really well: The AUC score is 0.99 which is ...
user avatar
  • 187
1 vote
1 answer
331 views

Interpreting vertical and horizontal parts of ROC curve

It's not clear to me how I can interpret vertical and horizontal parts of the ROC curve. What important information can I gain from this? This is a text from the book "Human-in-the-Loop Machine ...
user avatar
5 votes
1 answer
3k views

Micro Average vs Macro Average for Class Imbalance

I have a dataset consisting of around 30'000 data points and 3 classes. The classes are imbalanced (around 5'000 in class 1, 10'000 in class 2 and 15'000 in class 3). I'm building a convolutional ...
user avatar
  • 234
1 vote
1 answer
114 views

Is it possible to get an ROC curve using Relu activation?

Based on my understanding, given that Relu doesn't provide probabilities unlike Softmax, it's not possible to plot an ROC curve. However, is there some way to convert the output from a Relu to ...
user avatar
  • 113
1 vote
1 answer
46 views

How would you interpret the following ROC and PRC curves?

How would you interpret the following ROC and PRC curves? For example, I find it weird to understand that the precision actually increases at some point when recall increases as well. Is that even ...
user avatar
  • 125
3 votes
0 answers
63 views

Fast PR / ROC curves and corespondings AUPR / AUROC

I find myself in a position of calculating numerous PR / ROC curves and their associated area under the PR curves (AUPR) / area under the ROC curve (AUROC). Its is quite easy to perform those ...
user avatar
  • 2,175
0 votes
1 answer
79 views

Correctness of a ROC Curve

I've built a Decision Tree Classifier to practice with the sklearn library. My first task was to shuffle the iris dataset and split it keeping only the last 10 elements for the test. Then, after the ...
user avatar
2 votes
0 answers
1k views

AUC on ROC Curve near 1.0 for Multi-Class CNN but Precision/Recall are not perfect?

I am building a ROC Curve and calculating AUC for multi-class classification on the CIFAR-10 dataset using a CNN. My overall Accuracy is ~ 90% and my precision and recall are as follows: ...
user avatar
0 votes
1 answer
556 views

AUC ROC Curve multi class Classification

Here is the part of the code for ROC AUC Curve calculation for multiple classes. ...
user avatar
  • 93
2 votes
1 answer
88 views

What is the difference in plotting ROC curve with probability scores vs binary decisions

As the title reads what is the difference? Plotting the ROC w.r.t probability scores gives the stair cased version. But in my opinion I find that using binary decision is better because the ROC curve ...
user avatar
  • 21
1 vote
1 answer
56 views

Algorithm for Binary classification

I have a data set with huge number of features ( Approximately 3000) and a binary target variable . The reason I have too many features is because of one hot encoding many categorical variables in ...
user avatar