# Questions tagged [roc]

The tag has no usage guidance.

56 questions
Filter by
Sorted by
Tagged with
19 views

### ROC curve for multiclassification - results sound not correct

I'm working on a multiclassification task using LSTM algorithm, i generated my roc curve plots but they give scores like 1 , 0.99, 0.97 however i have an accuracy of 0.97, Precision 0.65, Sensitivity/...
42 views

### calculating tpr and fpr

While calculating the tpr and fpr, can I give both positive class probability or the actual predictions? it give different scores for me, please help me out
1 vote
70 views

### What is ROC curve based on for SVM?

I was studying about the ROC curves for Logistic regression. There is a threshold in this method that determines the classification. By changing this threshold we get different confusion matrices and ...
1k views

### Is it sensible to use the ROC curve with an KNN model? And if so why?

I am a beginner doing my first ML project. I am doing a binary supervised classification on an unbalanced dataset and want to use the ROC curve as a performance metric of my models. I am using ...
157 views

### Does it make sense to build a ROC curve for Naive Bayes classification?

These past days, in college, we have been learning about NaiveBayes. Since it's a classification algorithm, I was wondering if I could evaluate NaiveBayes models the same way (using the same metrics) ...
39 views

### What does precision-recall curve and ROC curve tell us abouth threshold invariance

Consider a binary classification problem. Intuitively, a value for the area under the curve (for both curves) very close to 1, shows that the curve is almost L-shaped. Thus, this means that the value ...
1 vote
83 views

### ROC Curve for model validation

Is there a general approach that the ROC curve can be used for to validate a model? My understanding is that we can use it to compare different threshold values to determine the best, or even see how ...
8 views

### Low value of area under PR and high value of a area under ROC interpretation?

I am still a rookie. I am training a random forest model and I am getting 0.27 for area under PR and 0.85 for area under ROC. My data was very imbalanced for the negative labels and I did perform a ...
72 views

### Smallest possible difference between AUC of two ranker [closed]

If there are 10 positive examples, and 90 negative examples in the test set, what is the smallest possible difference in AUC, between two rankers giving different AUC?
106 views

### What makes an ROC curve a curve and why do the values change?

I have a problem. I am currently looking at a classifier and I would like to examine this using an ROC curve as a metric. However, questions have arisen to which I can not find an answer. A ROC curve ...
1 vote
781 views

### Inverted ROC curve

I am using tidymodels package in R. Running random forest to classify three classes. There are about 8000 samples in total and 130 features. This is how the ROC curves look like. The predictions for ...
602 views

### Why I am having trouble plotting the AUC?

I am trying to plot the roc_auc curve however I am not getting any results. Any explanation here? Are there any problems with the number of data? Here is my example : ...
41 views

### How do I modify a Logistic Regression to target a specific point on the ROC curve?

From a conceptual standpoint I understand the trade off involved with the ROC curve. You can increase the accuracy of true positive predictions but you will be taking on more false positives and vise ...
284 views

### How do I compute the Weighted average ROC Curve?

So i have a multiclass problem and successfully computed the micro and macro average curves, how do I calculate the weighted value for each TPR and FPR?
1 vote
286 views

### How to draw each ROC curve of an SVM model with cross validation

I would like to make a graph like the following in python: That is, one curve for each fold. I have the following code where I use an SVM model to classify some data ...
305 views

### Should I be using y_pred or y_pred_proba for binary Classification?

I have a binary classification problem and i want to plot ROC/AUC curve, should I use ypred or ypred_proba
302 views

### ROC-AUC Imbalanced Data Score Interpretation

I have a binary response variable (label) 𝐵 in a dataset with around 50,000 observations. The training set is somewhat imbalanced with, 𝐵𝑖=1 making up about 33% of the observation's and 𝐵𝑖=0 ...
1 vote
174 views

### How to plot one graph of ROC curve for 4 separate ML model located in different python notebooks

if we have 4 different notebooks for different ML model results .. and we have to plot one ROC curve graph which shows the ROc of all 4 models. how can we do this this is my code in every notebook to ...
268 views

### Interpreting ROC curves across k-fold cross-validation

I have used a MARS model (multivariate adaptive regression splines) and I have used k fold cross validation for the evaluation of the model, obtaining the following graph: How would be the ...
33 views

### Can you get a very good AUC-ROC score despite predicting all rows to have the same probability?

On the test set of a binary classification problem, the p25, p50 and p75 of the predictions are very close to each other (e.g. 0.123). Is it possible that my model can achieve a high AUC-ROC (e.g. 0....
1 vote
16 views

### My data can be approximated with Normal mixture. How can I find the reasons and explain this behaviour?

I use DeLonge method to compare two ROC AUCS. The result of it is Z-score. Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
157 views

### What does it mean when roc curves intersect at a point?

I am working with a data set and I have obtained the following roc curve: As you can see, black and Asian ethnicity cross at one point (green and purple lines). Does this have any significance? Could ...
1 vote
16 views

### How to ensamble different ranking models?

I have trained two different models, which give a score to each data point. The score of the models it is not necessarily comparable. The score is used to give a ranking, and the performance is ...
1 vote
335 views

### Improving roc auc score when accuracy is good

I have got a binary classification problem with large dataset of dimensions (1155918, 55) Also dataset is fairly balanced of 67% Class 0 , 33% Class 1. I am getting test accuracy of 73% in test set ...
1 vote
14 views

### Does thereshold of classifier close to 0 make sense?

I have roc curve with AUC of 0.91. I applied the following function to determine the best threshold: ...
1k views

### Uncertainty about shape of ROC curve

I am working on a binary classification and the plotted ROC curves that I am using for evaluation together with AUC, have seemed strange to me. Here is an example. I understand that ROC is a visual ...
92 views

### What happens to auc when true positive rate grows

How does change in true positive rate affects AUC? Does increase of TPR lead to increase of AUC as well?
73 views

### Is roc auc graph better than roc auc score? If yes why?

This was asked in viva of my ML course. I answered yes but could not precisely explain why. By 'better' I mean whether geometric interpretation gives more information than just the numeric score.
353 views

### Use one-vs-rest ROC AUC to get threshold for each class

I have a Neural Network with 4 classes (completely balanced) where the recall is the following for each class (the class with the highest score from the network is chosen) $RE(C1) = 0.611650$ \$RE(C2) ...
1k views

### roc_auc_score from sk-learn gives error when test label vector with classes has only a subset of the whole set

I have an imbalanced dataset. Does it make sense to compute the roc-auc for the classifier I created in a holdout set? Here's very artificial MWE: ...
1 vote
153 views

### Identify which is the best point(s) for (ROC) curve(s)

This is an theorical question, so, I am looking for the point in a ROC Curve. And I got the idea, that different curves has different best point. So, I try to identify those points. For the yellow ...
882 views

### Identify optimal thresholds for one-vs-one/one-vs-rest ROC-curve for multiclass classification

Say I have a multiclass classification problem with N classes. I have trained a classifier on a training set, I use a validation set and a One-vs-rest ROC-curve to ...
1 vote
111 views

### If ROC is used to find a threshold, but AUC is threshold invariant, why use AUC?

Say I have a binary classifier. I calculate ROC to select an ideal threshold of say, 0.6. Then, I look at the AUC. But wait! If AUC doesn't change by selecting an 0.6 threshold, then what makes AUC ...
16 views

### How can I Determine a Treshold According to the Precision and Recall?

I am gettin these precision and recall values from my classifier and I want to determine a treshold for the test data. How can I determine that treshold? Is these values enough or something else is ...
150 views

### Does it make sense to repeat calculating AUC in logistic regression?

I have a question regarding logistic regression models and testing its skill. I am not quite sure if I understand correctly how the ROC Curve is established. When calculating the ROC curve, is a train ...
1 vote
330 views

### ROC and AUC curve for CNN multi-class classification problem

I have produced a convolutional neural network to classify images (malware images) into different classes/families. I have managed to produce a confusion matrix and classification report. My ...
1 vote
258 views

### Overall AUC higher than all "stratified" AUCs

For one of my binary classification models, I have observed this (Simpson's Rule-esque) paradox. The AUC on the test set as a whole is 0.8. Gender is one of the model's features. So I decided to ...
973 views

### Implementing the Trapezoid rule without the formula for the curve

I know that if I have some function f(x) that describes a curve, I can approximate the area under the curve using the trapezoid rule as follows: ...
3k views

### Can Micro-Average Roc Auc Score be larger than Class Roc Auc Scores

I'm working with an imbalanced data set. There are 11567 negative and 3737 positive samples in train data. There are 2892 negative and 935 positive samples in validation data. It is a binary ...
258 views

### At what stage are ROC curves used when building machine learning model?

When developing a machine learning model, at what stage are ROC curve with AUC used? Typically I have three data sets train - ...
1 vote
313 views

### Best practice to select precision vs. recall threshold for a binary classifier

I have a logistic regression model in Scikit-Learn doing a binary classification. Looking at the Roc curve for the classifier I can see that it performs really well: The AUC score is 0.99 which is ...
925 views

### Interpreting vertical and horizontal parts of ROC curve

It's not clear to me how I can interpret vertical and horizontal parts of the ROC curve. What important information can I gain from this? This is a text from the book "Human-in-the-Loop Machine ...
5k views

### Micro Average vs Macro Average for Class Imbalance

I have a dataset consisting of around 30'000 data points and 3 classes. The classes are imbalanced (around 5'000 in class 1, 10'000 in class 2 and 15'000 in class 3). I'm building a convolutional ...
1 vote
180 views

### Is it possible to get an ROC curve using Relu activation?

Based on my understanding, given that Relu doesn't provide probabilities unlike Softmax, it's not possible to plot an ROC curve. However, is there some way to convert the output from a Relu to ...
1 vote
59 views

### How would you interpret the following ROC and PRC curves?

How would you interpret the following ROC and PRC curves? For example, I find it weird to understand that the precision actually increases at some point when recall increases as well. Is that even ...
71 views

### Fast PR / ROC curves and corespondings AUPR / AUROC

I find myself in a position of calculating numerous PR / ROC curves and their associated area under the PR curves (AUPR) / area under the ROC curve (AUROC). Its is quite easy to perform those ...
137 views

### Correctness of a ROC Curve

I've built a Decision Tree Classifier to practice with the sklearn library. My first task was to shuffle the iris dataset and split it keeping only the last 10 elements for the test. Then, after the ...
2k views

### AUC on ROC Curve near 1.0 for Multi-Class CNN but Precision/Recall are not perfect?

I am building a ROC Curve and calculating AUC for multi-class classification on the CIFAR-10 dataset using a CNN. My overall Accuracy is ~ 90% and my precision and recall are as follows: ...