# Questions tagged [auc]

The tag has no usage guidance.

49 questions
Filter by
Sorted by
Tagged with
21 views

### How can I compute the AUC by using Gaussian Mixture Model?

By using this code, can I compute the AUC: ...
17 views

### How to measure multi-label multi-class accuracy

I have a model that has multi-label multi-class targets Example Age Height Weight Mark Distance Red Yellow Green Blue Black White 14 160 62 78 103 0 1 1 1 1 0 56 177 90 99 363 1 1 0 0 0 0 32 179 ...
20 views

### How to interpret the Precision Recall AUC

The ROC AUC has an intuitive interpretation: the probability that the score of a randomly sampled 1-labeled item will be higher than a randomly sampled 0-labeled item. Is there a similar ...
40 views

### Clarification on tf.keras.metrics.AUC

Question Does the AUC metric calculates the area of ROC or PR? Background tf.keras.metrics....
14 views

### Reliability of tf.keras.metrics.AUC metric calculation

Please help understand how accurate or reliable tf.keras.metrics.AUC metric is. It looks tf.keras.metrics.AUC is not actually ...
34 views

### If ROC is used to find a threshold, but AUC is threshold invariant, why use AUC?

Say I have a binary classifier. I calculate ROC to select an ideal threshold of say, 0.6. Then, I look at the AUC. But wait! If AUC doesn't change by selecting an 0.6 threshold, then what makes AUC ...
77 views

### AUC higher than accuracy in multi-class problem

I stumbled upon a 3-class classification problem where all compared classifiers yield a higher AUC than accuracy (usually around 10% higher). This happens both when the dataset is balanced or slightly ...
40 views

### Does it make sense to repeat calculating AUC in logistic regression?

I have a question regarding logistic regression models and testing its skill. I am not quite sure if I understand correctly how the ROC Curve is established. When calculating the ROC curve, is a train ...
33 views

### Inconsistancy in Sklearn SVM predict() and predict_proba()

Actually I have two questions. One of them is related the bug of sklearn SVM model and the other one is about ROC-AUC score. My first question is related to ROC-AUC score but also includes a bug ...
17 views

### What is the appropriate statistical test to compare the MAUC scores from two machine learning classifiers?

I would like to compare the scores of two multi-class classifiers. I have calculated the MAUC score for each of the algorithms, and now I want to see whether there is a statistical difference between ...
20 views

### How to optimize AUPRC for imbalanced data given a precision or recall bias?

My general understanding is that when optimizing a model in an imbalanced class case with a small preferred target class one should optimize first for a model with the best AUPRC (assuming one doesn't ...
35 views

### ROC and AUC curve for CNN multi-class classification problem

I have produced a convolutional neural network to classify images (malware images) into different classes/families. I have managed to produce a confusion matrix and classification report. My ...
22 views

### How to interpret stagnant validation curve

I'm new to deep learning, so I'm just learning how to interpret my models. I'm creating a mixed-convolutional neural net to classify melanoma images. Here's the model structure: ...
8 views

### Model performance in different snapshots varying

I am trying to solve this problem. A medical representative needs to visit some doctors' clinics and for that a model will generate probability scores for visiting a clinic. I ma using a tree based ...
32 views

### LightGBM model improvement when the focus is on probability prediction

I am building a binary classifier using LightGBM. The goal is not to predict the outcome as such, but rather to predict the probability of the target even. To be more specific, it's more about ranking ...
65 views

### Overall AUC higher than all "stratified" AUCs

For one of my binary classification models, I have observed this (Simpson's Rule-esque) paradox. The AUC on the test set as a whole is 0.8. Gender is one of the model's features. So I decided to ...
83 views

### Implementing the Trapezoid rule without the formula for the curve

I know that if I have some function f(x) that describes a curve, I can approximate the area under the curve using the trapezoid rule as follows: ...
92 views

### At what stage are ROC curves used when building machine learning model?

When developing a machine learning model, at what stage are ROC curve with AUC used? Typically I have three data sets train - ...
108 views

### Best practice to select precision vs. recall threshold for a binary classifier

I have a logistic regression model in Scikit-Learn doing a binary classification. Looking at the Roc curve for the classifier I can see that it performs really well: The AUC score is 0.99 which is ...
29 views

### When to use AUROC OvR vs. AUROC OvO?

For multiclass problems, there are 2 versions of the AUROC metric: the AUROC OvR and AUROC OvO. Does anyone know in what particular cases we would use AUROC OvR vs. AUROC OvO? In the general academic ...
79 views

### Is the PR AUC invariant under label flip?

The ROC-AUC curve is invariant under a flip of the labels. I don't know if its a famous result so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not ...
759 views

### Micro Average vs Macro Average for Class Imbalance

I have a dataset consisting of around 30'000 data points and 3 classes. The classes are imbalanced (around 5'000 in class 1, 10'000 in class 2 and 15'000 in class 3). I'm building a convolutional ...
34 views

### Fast PR / ROC curves and corespondings AUPR / AUROC

I find myself in a position of calculating numerous PR / ROC curves and their associated area under the PR curves (AUPR) / area under the ROC curve (AUROC). Its is quite easy to perform those ...
521 views

### AUC on ROC Curve near 1.0 for Multi-Class CNN but Precision/Recall are not perfect?

I am building a ROC Curve and calculating AUC for multi-class classification on the CIFAR-10 dataset using a CNN. My overall Accuracy is ~ 90% and my precision and recall are as follows: ...
285 views

### AUC ROC Curve multi class Classification

Here is the part of the code for ROC AUC Curve calculation for multiple classes. ...
47 views

### Algorithm for Binary classification

I have a data set with huge number of features ( Approximately 3000) and a binary target variable . The reason I have too many features is because of one hot encoding many categorical variables in ...
18 views

### How to compute a confidence interval for AUC?

I found that in results of several binary classification problems, people report an AUC value together with a CI. I wonder how those CIs are computed. Is there a close-formed formula to compute them ...
515 views

### Don't understand why I get an inverse ROC curve for SVM (Python)

I build an SVM classifier but get an inverse ROC curve. The AUC is only 0.08. I've used the same datasets to build a Logistic Regression classifier and a Decision Tree classifier, and the ROC curves ...
18 views

### confused AUC ROC score [duplicate]

I am working on binary classification problem, I try to evaluate the performance of some classification algorithms (LR,Decission Tree , Random forest ...). I am using a 10 fold cross-validation ...
130 views

### Confused AUC ROC score

I am working on binary classification problem, I try to evaluate the performance of some classification algorithms (LR,Decission Tree , Random forest ...). I am using a cross validation technique (to ...
39 views

### XG Boost result interpretation for unbalanced datasets (Accuracy & AUCROC)

My dataset is of shape – 5621*8 (binary classification) Label/target : Success (4324, 77 %) & Not success (1297, 23 %) (success and Not success were been ...
134 views

### Main options on how to deal with imbalanced data

As far as I can tell, broadly speaking, there are three ways of dealing with binary imbalanced datasets: Option 1: Create k-fold Cross-Validation samples randomly (or even better create k-fold ...
50 views

### Orange ROC analysis widget [closed]

ROC analysis widget has an option to change prior target class probability. I'd like to know, when and how it should be used. Playing with it changes the slope of iso-performance line. Each classifier ...
248 views

### Getting lower performance metrics when using GridSearchCV

I have defined an XGBoost model and would like to tune some of its hyperparameters. I am using GridSearchCV to find the best params. However, I also tried to fit the model on the entire training ...
464 views

### How to compute AUC in gridsearchSV (multiclass problem)

I'm working on a multiclass classification problem, comparing results from SVM and Random Forest classificators. I would like to use gridsearchCV for hyperparameters tuning and find that AUC is the ...
225 views

### Confusion matrix and ROC AUC curves are not in sync

I created a classification model with three target classes and created a confusion matrix to measure the accuracy, here is the matrix code ...
5k views

### Plotting ROC & AUC for SVM algorithm

Towards , the end of my program, I have the following code. model = svm.OneClassSVM(nu=nu, kernel='rbf', gamma=0.00001) model.fit(train_data) Output ...
88 views

### Confusion Matrix and AUC in univariate Anomaly Detection

In the code I upon a csv file which only has one column. The data in there in not that important just normal numbers. ...
4k views

### AUC-ROC for Multi-Label Classification

Hey guys I'm currently reading about AUC-ROC and I have understood the binary case and I think that I understand the multi-classification case. Now I'm a bit confused on how to generalize it to the ...
15 views

### Connection between prob output LogisticReg/SVM and ROC

I have the following ROC generated using LPOCV and Logistic regression or SVM (l2 norm). Now, let's say I have a test set containing 10 patients and I get that the probabilities of those patients to ...
204 views

### Imbalanced dataset - Positive majority class

My dataset consists of 150 patients where 50 are controls/healthy (negative) and 100 are sick (positive). If I want my model to have high sensitivity at hight specificity, in other words to have low ...
64 views

### Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed Recursive Feature Elimination (RFE) and stratified cross validation to reduce the features to 15 and I get ...
40 views

### How much can the AUC improve comparing the raw dataset and the feature engineered dataset?

Let's say I put the following two datasets in the best possible model (same model for both): A raw dataset, the variables as they came just from the query. A feature-engineered dataset, with hundreds ...
167 views

### What is AUC - ROC Curve?

AUC - ROC curve is a performance measurement for classification problem at various thresholds settings. ROC is a probability curve and AUC represents degree or measure of separability. Is Roc the ...
13 views

### Truncating float/doubles for reproducibility

I deploy machine learning models (typically GPU) to a variety of environments. I work sort of at the edge of ML R&D and devops, so I am really big into reproducibility, and one thing that drives ...
345 views

### How to create an roc plot and calculate AUC for an svm (that does not return probabilities)?

I have some SVM classifier outputting final classifications for every sample in the test set, something like 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1 and so on. The "...
247 views

### ROC-AUC curve as metric for binary classifier without machine learning algorithm

Maybe it's a trivial question, but I'm a bit confused right now... I'll explain: I've some elements in my data, each with a value between 0 and 1 and an associated label (1, 0). I need to test some ...