The 2024 Developer Survey results are live! See the results

# Tag Info

Accepted

### Micro Average vs Macro Average for Class Imbalance

The question is actually about understanding what it means to "take imbalance into account": Micro-average "takes imbalance into account" in the sense that the resulting ...
• 25.6k
Accepted

### Is the PR AUC invariant under label flip?

I propose to look at this a bit differently. The different classification status of the two classes are related as follows: $TP_1=TN_0$ $FP_1=FN_0$ $FN_1=FP_0$ $TN_1=TP_0$ From there we have: ...
• 25.6k

### What is AUC - ROC Curve?

Related concepts, but not the same. ROC-receiver operating curve AUC area under the curve Thank this post for explanation : Abbreviations AUC = Area Under the Curve. AUROC = Area Under the ...
• 5,719

### AUC-ROC for Multi-Label Classification

I am not sure if I understand your thinking, but my understanding is following - maybe it can help you see it from another perspective. The multi-label classification problem with ...
• 263

### How much can the AUC improve comparing the raw dataset and the feature engineered dataset?

Yes, the performance can vary a lot using feature engineering. Example: suppose a dataset where the response variable $y$ is true if $x$ is odd. ...
• 25.6k

### How to choose between different models with similar results? RF, GLM and XGBoost

Should I do more performance measurements? Which ones and why? Generally speaking, the choice of performance metric should be informed by your goals and the characteristics of your dataset. If you ...
• 2,744

### Truncating float/doubles for reproducibility

I'm not expert in this but as far as I know the proper way to test for equality modulo floating point imprecision is to compare the differences of the two values, i.e. instead of: ...
• 25.6k

### How much can the AUC improve comparing the raw dataset and the feature engineered dataset?

I would say that the best possible model for the raw data would derive all the meaningful features that you would have created from the data anyway. And I would say that the best possible model for ...
• 1,051

### Plotting ROC & AUC for SVM algorithm

If you are performing a binary classification task then the following code might help you. from sklearn.model_selection import GridSearchCV for hyper-parameter ...
• 121

### Plotting ROC & AUC for SVM algorithm

The ROC curve requires probability estimates (or at least a realistic rank-ordering), which one-class SVM doesn't really try to produce. https://stats.stackexchange.com/a/99179/232706 https://...
• 12k

### Confusion Matrix and AUC in univariate Anomaly Detection

It's difficult to answer precisely without knowing the data and the task. Assuming it's a single column of values with no order involved, it boils down to finding the optimal threshold to separate ...
• 25.6k

### Don't understand why I get an inverse ROC curve for SVM (Python)

One potential fix is to remove max_iter = 12 (which would set it to the scikit learn default of max_iter=-1). Using such a low ...
• 5,450

### Don't understand why I get an inverse ROC curve for SVM (Python)

For any classification problem if AUC<0.5, you are not performing better than random(0.5). Reason could be: Your classifier is over-fitted on the training set and performs very poorly on the test ...
• 1,511

### How to compute AUC in gridsearchSV (multiclass problem)

There are actually several flavors of AUC you can now use with multiclass evaluation: 'roc_auc_ovo' 'roc_auc_ovo_weighted' 'roc_auc_ovr' 'roc_auc_ovr_weighted' These also work with ...
• 240
Accepted

### AUC ROC Curve multi class Classification

df = pd.get_dummies(pred1) df.insert(loc=2,column='2',value=0) #print(df) add this before the for loop and instead of using pd.get_dummies(y_test) use only df
• 158

### Does it make sense to repeat calculating AUC in logistic regression?

The calculation of ROC curve and the AUC based off of that curve is simply a comparison of the predictions from your model (logistic regression) and the actual values on some set of data. This can ...
Accepted

### Overall AUC higher than all "stratified" AUCs

AUC can be defined as $P(X_1 > X_0)$ where $X_1$ is the score of a randomly chosen positive instance and $X_0$ is the score of a randomly chosen negative instance. Like in Simpson's "paradox&...
• 1,254
Accepted

### Is roc auc graph better than roc auc score? If yes why?

Yes, the graph contains information that the AUC number alone does not have. It is most interesting when comparing 2+ models that have very close AUC numbers. The graph can tell you that one model ...
• 1,169

### Area Under the Precision Recall Curve

What happens is something like this: When the threshold is very high, with only very few instances predicted as positive, the precision is around 0.5. The recall is very low, since only a small ...
• 25.6k

### Why is Precision-Recall AUC different from Average Precision score?

I have tried to generate a dataset with a 95:5 class imbalance, train a RandomForestClassifier model, and calculate AUPRC and AUC-ROC and Average Precision (AP) ...
• 400
1 vote

### If ROC is used to find a threshold, but AUC is threshold invariant, why use AUC?

Roughly speaking, it depends on your purpose. AUC is for mathematical purpose (roughly speaking). It is a characteristic of the quality of your model. It depends on your data and your skills as model-...
• 1,522
1 vote
Accepted

### AUC higher than accuracy in multi-class problem

I think the question can se splitted to three parts. The first part is about comparing accuracy and AUC of the same model, the second is about comparing models and the third is about multi-class ...
• 806
1 vote
Accepted

### Implementing the Trapezoid rule without the formula for the curve

You're assuming that the points are equally spaced along the fpr axis, which is generally not true. See e.g. the "Uniform grid" vs "Nonuniform grid&...
• 12k
1 vote

### At what stage are ROC curves used when building machine learning model?

The ROC-AUC curves are used to find the best threshold that optimizes True Positive Rate vs False Positive Rate. Using it in a K-Fold cross-validation is a good practice to determine the best ...
• 1,415
1 vote
Accepted

### Algorithm for Binary classification

First thing that comes to my mind is to do different encodings. There are some ways to deal with high cardinality categorical data such as: Label Encoding or the famous target encoding. Before ...
• 6,302
1 vote

### How to compute AUC in gridsearchSV (multiclass problem)

Metrics are independent from ML algorithms, so it doesn't matter which algorithms did you use. To calculate multiclass AUC you could use lib pRoc in R or use code this link(in Python). Sources: ...
• 1,373
1 vote
Accepted

### Confused AUC ROC score

Oh, I think I've finally got it. It's just an averaging problem: for each fold in your k-fold cross-validation, you get perfect auROC, but at the default threshold of 0.5 your hard classifiers (for ...
• 12k
1 vote
Accepted

### XG Boost result interpretation for unbalanced datasets (Accuracy & AUCROC)

With Success already being the larger class, you probably shouldn't be using a scale_pos_weight larger than one: you want to scale the positive class's contribution ...
• 12k
1 vote

### Main options on how to deal with imbalanced data

Imbalanced class means the count of one class is too low compared to the count of other Class. This means Model will have little opportunity to learn the minority Class. We have these option to ...
• 5,644
1 vote

### Main options on how to deal with imbalanced data

your resume is quite good, but I'm not comfortable in dividing the broad discussion to those three more or less sharply separated roads. But indeed, often a technique similar to one of those is chosen....
• 1,774

Only top scored, non community-wiki answers of a minimum length are eligible