Questions tagged [auc]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

Why is ROC-AUC usually shown in GNN papers

In various graph neural network (GNNs) papers, the ROC-AUC metric is usually shown alone without considering F1 or Accuracy. Is there a reason for that? What does it say about two models 1 and 2 with ...
user avatar
0 votes
2 answers
54 views

Should I be using y_pred or y_pred_proba for binary Classification?

I have a binary classification problem and i want to plot ROC/AUC curve, should I use ypred or ypred_proba
user avatar
  • 27
0 votes
0 answers
9 views

How are ROC curves constructed? [duplicate]

I would like to understand how to build a ROC curve of a model. For example, if we would like to draw it by hand, what steps should we do? Thank you.
user avatar
  • 302
1 vote
0 answers
18 views

How can I calculate de AUC PR of my classifiers in a multiclass scenario?

I'm developing image classifiers in a context with 25k images and 50 classes. The dataset is imbalanced. Some papers recommend AUC PR for comparing the performance of my classifiers in this setting. ...
user avatar
  • 127
3 votes
1 answer
46 views

Area Under the Precision Recall Curve

I have got the following Precision Recall Curve for a classifier I built using AutoML. Most of the Precisio-Recall curves tend to start from (0, 1) go towards (1,0). But mine is the opposite. But I ...
user avatar
  • 151
1 vote
0 answers
15 views

My data can be approximated with Normal mixture. How can I find the reasons and explain this behaviour?

I use DeLonge method to compare two ROC AUCS. The result of it is Z-score. Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
user avatar
1 vote
0 answers
12 views

How to ensamble different ranking models?

I have trained two different models, which give a score to each data point. The score of the models it is not necessarily comparable. The score is used to give a ranking, and the performance is ...
user avatar
0 votes
0 answers
29 views

Fashion Compatibility Performance Evaluation: High in AUC but Low in FITB

I am a newbie in deep learning field. Still trying to understand how this works. But now I am working on fashion compatibility prediction. The most well-known performance evaluation in this task is ...
user avatar
  • 1
0 votes
0 answers
22 views

Model Selection when there is trade-off

This is one of my model variants. It achieves an AUC score of 0.73. Another one of my model variants achieves an AUC score of 0.7265. Below is the the confusion matrix - Like many problems, the ...
user avatar
  • 167
1 vote
1 answer
12 views

Does thereshold of classifier close to 0 make sense?

I have roc curve with AUC of 0.91. I applied the following function to determine the best threshold: ...
user avatar
  • 13
0 votes
0 answers
467 views

Xgboost Multiclass evaluation Metrics

Im training an Xgb Multiclass problem, but im having doubts about my evaluation metrics, heres my code + output ...
user avatar
0 votes
1 answer
38 views

What happens to auc when true positive rate grows

How does change in true positive rate affects AUC? Does increase of TPR lead to increase of AUC as well?
user avatar
  • 143
0 votes
0 answers
37 views

Is it possible to use roc auc metric in uplift modeling (class transformatio approach)

I do not understand why in uplift modeling (Class Transformation approach) not used ROC AUC score for changed target Z. I have a problem with a task where I tried to use this approach, but ROC AUC ...
user avatar
2 votes
3 answers
54 views

Is roc auc graph better than roc auc score? If yes why?

This was asked in viva of my ML course. I answered yes but could not precisely explain why. By 'better' I mean whether geometric interpretation gives more information than just the numeric score.
user avatar
  • 23
0 votes
0 answers
26 views

Threshold moving for classification - how can i assess the cost?

I have a binary classifier. When i used my model to make predictions about 4k out of 10k were predicted to be "Rich". I am predicting affluence. Normally in classification the cut off to ...
user avatar
  • 466
1 vote
0 answers
28 views

Linear combination of features reverses importance of all features

I am trying with a logistic model with 2 features independently or with linear combination, but in the linear combination, combining these features would reverse importance through significance levels ...
user avatar
0 votes
1 answer
73 views

Interpreting evaluation metrics with threshold/cutoff

I was doing churn prediction for a company. I've got the following results by applying 3 classifier. Model Accuracy AUC Logistic Regression 0.671 0.736 Decision Tree (pruned) 0.681 0.665 Decision ...
user avatar
  • 23
0 votes
0 answers
20 views

Different AUC values for xgb and sklearn built in functions [duplicate]

The model is trained with early stopping on a validation set: ...
user avatar
  • 133
0 votes
0 answers
170 views

How can I compute the AUC by using Gaussian Mixture Model?

By using this code, can I compute the AUC: ...
user avatar
  • 21
1 vote
1 answer
78 views

How to measure multi-label multi-class accuracy

I have a model that has multi-label multi-class targets Example Age Height Weight Mark Distance Red Yellow Green Blue Black White 14 160 62 78 103 0 1 1 1 1 0 56 177 90 99 363 1 1 0 0 0 0 32 179 ...
user avatar
  • 497
1 vote
1 answer
26 views

How to interpret the Precision Recall AUC

The ROC AUC has an intuitive interpretation: the probability that the score of a randomly sampled 1-labeled item will be higher than a randomly sampled 0-labeled item. Is there a similar ...
user avatar
  • 111
0 votes
1 answer
275 views

Clarification on tf.keras.metrics.AUC

Question Does the AUC metric calculates the area of ROC or PR? Background tf.keras.metrics....
user avatar
  • 483
0 votes
0 answers
49 views

Reliability of tf.keras.metrics.AUC metric calculation

Please help understand how accurate or reliable tf.keras.metrics.AUC metric is. It looks tf.keras.metrics.AUC is not actually ...
user avatar
  • 483
1 vote
2 answers
51 views

If ROC is used to find a threshold, but AUC is threshold invariant, why use AUC?

Say I have a binary classifier. I calculate ROC to select an ideal threshold of say, 0.6. Then, I look at the AUC. But wait! If AUC doesn't change by selecting an 0.6 threshold, then what makes AUC ...
user avatar
1 vote
1 answer
344 views

AUC higher than accuracy in multi-class problem

I stumbled upon a 3-class classification problem where all compared classifiers yield a higher AUC than accuracy (usually around 10% higher). This happens both when the dataset is balanced or slightly ...
user avatar
0 votes
1 answer
77 views

Does it make sense to repeat calculating AUC in logistic regression?

I have a question regarding logistic regression models and testing its skill. I am not quite sure if I understand correctly how the ROC Curve is established. When calculating the ROC curve, is a train ...
user avatar
1 vote
0 answers
27 views

What is the appropriate statistical test to compare the MAUC scores from two machine learning classifiers?

I would like to compare the scores of two multi-class classifiers. I have calculated the MAUC score for each of the algorithms, and now I want to see whether there is a statistical difference between ...
user avatar
  • 31
1 vote
0 answers
177 views

ROC and AUC curve for CNN multi-class classification problem

I have produced a convolutional neural network to classify images (malware images) into different classes/families. I have managed to produce a confusion matrix and classification report. My ...
user avatar
  • 125
0 votes
0 answers
31 views

How to interpret stagnant validation curve

I'm new to deep learning, so I'm just learning how to interpret my models. I'm creating a mixed-convolutional neural net to classify melanoma images. Here's the model structure: ...
user avatar
  • 113
1 vote
1 answer
116 views

Overall AUC higher than all "stratified" AUCs

For one of my binary classification models, I have observed this (Simpson's Rule-esque) paradox. The AUC on the test set as a whole is 0.8. Gender is one of the model's features. So I decided to ...
user avatar
0 votes
1 answer
347 views

Implementing the Trapezoid rule without the formula for the curve

I know that if I have some function f(x) that describes a curve, I can approximate the area under the curve using the trapezoid rule as follows: ...
user avatar
3 votes
1 answer
184 views

At what stage are ROC curves used when building machine learning model?

When developing a machine learning model, at what stage are ROC curve with AUC used? Typically I have three data sets train - ...
user avatar
  • 130
1 vote
0 answers
159 views

Best practice to select precision vs. recall threshold for a binary classifier

I have a logistic regression model in Scikit-Learn doing a binary classification. Looking at the Roc curve for the classifier I can see that it performs really well: The AUC score is 0.99 which is ...
user avatar
  • 187
2 votes
2 answers
210 views

Is the PR AUC invariant under label flip?

The ROC-AUC curve is invariant under a flip of the labels. I don't know if its a famous result so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not ...
user avatar
5 votes
1 answer
3k views

Micro Average vs Macro Average for Class Imbalance

I have a dataset consisting of around 30'000 data points and 3 classes. The classes are imbalanced (around 5'000 in class 1, 10'000 in class 2 and 15'000 in class 3). I'm building a convolutional ...
user avatar
  • 234
3 votes
0 answers
62 views

Fast PR / ROC curves and corespondings AUPR / AUROC

I find myself in a position of calculating numerous PR / ROC curves and their associated area under the PR curves (AUPR) / area under the ROC curve (AUROC). Its is quite easy to perform those ...
user avatar
  • 2,145
2 votes
0 answers
976 views

AUC on ROC Curve near 1.0 for Multi-Class CNN but Precision/Recall are not perfect?

I am building a ROC Curve and calculating AUC for multi-class classification on the CIFAR-10 dataset using a CNN. My overall Accuracy is ~ 90% and my precision and recall are as follows: ...
user avatar
0 votes
1 answer
477 views

AUC ROC Curve multi class Classification

Here is the part of the code for ROC AUC Curve calculation for multiple classes. ...
user avatar
  • 93
1 vote
1 answer
55 views

Algorithm for Binary classification

I have a data set with huge number of features ( Approximately 3000) and a binary target variable . The reason I have too many features is because of one hot encoding many categorical variables in ...
user avatar
1 vote
0 answers
19 views

How to compute a confidence interval for AUC?

I found that in results of several binary classification problems, people report an AUC value together with a CI. I wonder how those CIs are computed. Is there a close-formed formula to compute them ...
user avatar
2 votes
2 answers
1k views

Don't understand why I get an inverse ROC curve for SVM (Python)

I build an SVM classifier but get an inverse ROC curve. The AUC is only 0.08. I've used the same datasets to build a Logistic Regression classifier and a Decision Tree classifier, and the ROC curves ...
user avatar
  • 31
0 votes
0 answers
19 views

confused AUC ROC score [duplicate]

I am working on binary classification problem, I try to evaluate the performance of some classification algorithms (LR,Decission Tree , Random forest ...). I am using a 10 fold cross-validation ...
user avatar
  • 103
5 votes
2 answers
181 views

Confused AUC ROC score

I am working on binary classification problem, I try to evaluate the performance of some classification algorithms (LR,Decission Tree , Random forest ...). I am using a cross validation technique (to ...
user avatar
  • 103
0 votes
1 answer
48 views

XG Boost result interpretation for unbalanced datasets (Accuracy & AUCROC)

My dataset is of shape – 5621*8 (binary classification) Label/target : Success (4324, 77 %) & Not success (1297, 23 %) (success and Not success were been ...
user avatar
  • 155
2 votes
2 answers
239 views

Main options on how to deal with imbalanced data

As far as I can tell, broadly speaking, there are three ways of dealing with binary imbalanced datasets: Option 1: Create k-fold Cross-Validation samples randomly (or even better create k-fold ...
user avatar
  • 21
1 vote
0 answers
63 views

Orange ROC analysis widget [closed]

ROC analysis widget has an option to change prior target class probability. I'd like to know, when and how it should be used. Playing with it changes the slope of iso-performance line. Each classifier ...
user avatar
  • 21
0 votes
2 answers
402 views

Getting lower performance metrics when using GridSearchCV

I have defined an XGBoost model and would like to tune some of its hyperparameters. I am using GridSearchCV to find the best params. However, I also tried to fit the model on the entire training ...
user avatar
  • 121
1 vote
2 answers
788 views

How to compute AUC in gridsearchSV (multiclass problem)

I'm working on a multiclass classification problem, comparing results from SVM and Random Forest classificators. I would like to use gridsearchCV for hyperparameters tuning and find that AUC is the ...
user avatar
  • 13
1 vote
1 answer
339 views

Confusion matrix and ROC AUC curves are not in sync

I created a classification model with three target classes and created a confusion matrix to measure the accuracy, here is the matrix code ...
user avatar
-1 votes
2 answers
12k views

Plotting ROC & AUC for SVM algorithm

Towards , the end of my program, I have the following code. model = svm.OneClassSVM(nu=nu, kernel='rbf', gamma=0.00001) model.fit(train_data) Output ...
user avatar
  • 585