8 votes
Accepted

Does a random classifier have a diagonal ROC (received operator characteristic) curve even when the data is biased toward negatives?

Yes, it will still be diagonal. The random model baseline predicts a probability between (0, 1) for each item. Recall the definitions of TPR and FPR TPR: $\frac{TP}{...
Karl's user avatar
  • 481
5 votes

How to choose the right threshold for binary classification?

In short, you should be the judge of that: depending on the precision (interested to minimise "false alarms/FP") and recall (interested to minimise "missed positives/FN") you want ...
hH1sG0n3's user avatar
  • 2,008
5 votes
Accepted

Which machine learning algorithms are more suitable for binary classification?

If you want to be highly literal, logistic regression is excellent for binary classes but completely inappropriate for $3+$ classes. No worries: there is multinomial logistic regression, the theory of ...
Dave's user avatar
  • 3,698
4 votes
Accepted

Different result of classification with same classifier and same input parameters

From sklearns random forest documentation: random_state int, RandomState instance or None, default=None Controls both the randomness of the bootstrapping of the samples used when building trees (if <...
Adrian B's user avatar
  • 198
3 votes

ROC-AUC Imbalanced Data Score Interpretation

if the model just guessed šµš‘–=0 it would also achieve a ROC-AUC score of 0.67. This is incorrect. The ROC curve is defined by varying a decision threshold, and so requires a probability or other ...
Ben Reiniger's user avatar
  • 11.7k
3 votes

How to deal with Different Shapes of X_train and X_test after OneHotEncoding?

The issue that you are running into is because you are using the fit_transform method on both your training and test dataset. The correct way of using a transformer ...
Oxbowerce's user avatar
  • 7,307
3 votes
Accepted

Python xgboost predicting future events

For first issue, Please check the result after making column order of prediction dataset same as training dataset(Date,Month,Day,...) You can check this link You have specified objective of regression ...
Udaya Unnikrishnan's user avatar
3 votes
Accepted

What could go wrong if I sample before classification?

Search for imbalanced data here and on cross-validated. There is a lot of discussion on techniques, pros and cons. Lets say it depends. But using "wrong" in the title may be too strong. ...
Craig's user avatar
  • 944
3 votes
Accepted

How are scores calculated for each class of binary classification

Your confusion matrix does not correspond to your classification report. Also the matrix that you show is not standard: the labels "True Positive" and "True negative" are ...
Erwan's user avatar
  • 25.2k
3 votes
Accepted

How do you add negative class sample for binary classification?

I think it is important to think about the application of that classifier and get the negative class images to be from a similar distribution as will be your application. For example if you want to ...
Tomas P's user avatar
  • 146
3 votes

Binary Classification with Very Small Dataset (<40 samples)

Maybe with data as small as 36 and only 3 positive samples the way is try to do a simple rule instead of a model, because it looks like a machine learning approach wouldn't work. But if you really ...
Allan's user avatar
  • 131
3 votes

How can SHAP feature importance be greater than 1 for a binary classification problem?

First, SHAP values are not directed translated as probabilities, they are marginal contributions for model's output. As explained in this post, we can't interpret SHAP values from raw predictions. ...
Victor Oliveira's user avatar
3 votes

ROC curve for a perfect model, why is AUC 1.0?

The difficulty with ROC curves is to understand what happens when the threshold varies. There is no summing, the curve only depends on how many instances have TP/FP/TN/FN status for every threshold. A ...
Erwan's user avatar
  • 25.2k
2 votes
Accepted

Binary Classification with Imbalanced Target

If you don't have any way to obtain negative instances, the standard option is one-class classification: one-class classification (OCC), also known as unary classification or class-modelling, tries ...
Erwan's user avatar
  • 25.2k
2 votes

How are scores calculated for each class of binary classification

First I will try to explain it in words - for me it always help to grasp the idea. So class precision supposed to measure how precise is your prediction given the class you predicted. For example lets ...
Kulikr's user avatar
  • 101
2 votes

Which machine learning algorithms are more suitable for binary classification?

The answer is that there is no one answer. The choice of machine learning algorithms - whether for binary or multi-class classification - very much depends upon your data and your application: how ...
Todd's user avatar
  • 36
2 votes

How to combine binary classification with patient stratification?

I started writing this as a comment but I realized that I have too many things to say... I'm not sure that it's a proper answer either but hopefully it's useful: I'm not really sure that I understand ...
Erwan's user avatar
  • 25.2k
2 votes
Accepted

how to interpret precsion recall value in binary classification of scikit-learn

Mostly when you are doing the binary class classification you are mostly interested in predicting the 1s. Some example of binary classification like load default or not( default 1, else 0), whether a ...
Ashwiniku918's user avatar
  • 1,964
2 votes

Creating Dataset for Classification, How much balanced a good dataset should be?

The answer to such wide questions always have the same answer - it depends. Since I don't know the exact ratio of the four classes, I'll mention a few important points that can help you decide how to ...
MetaInformation's user avatar
2 votes

Meaningfully compare target vs observed TPR & FPR

Your points totally make sense, as it is key to monitor models performance in an MLOps strategy. About these points, I would say: the metric you might want to monitor to measure your model ...
German C M's user avatar
  • 2,696
2 votes
Accepted

Binary Classification with Very Small Dataset (<40 samples)

I'm not sure this will be a comprehensive answer but an opinion to give a push to the reasoning. There are only 3 negative cases. I could create a custom cross validation scheme: create a test case ...
Sergey Skripko's user avatar
2 votes

The best Python library to build decision tree on binary inputs

Why don't you start with sklearn. You can set maximum number of elements in leaf Minimum element at root Decision Tree takes batch algorithm. You can input 20k samples all at once. The CPU config ...
amol goel's user avatar
  • 341
2 votes
Accepted

Finding research papers for a dataset

Super important question. The reason is that this is not the original source. If you go to the data -> meta data -> sources, you can see the source is: ...
GooJ's user avatar
  • 435
2 votes

Timing of applying random oversampling on the dataset

Current answers are roughly correct, but miss the main split of when you would use it. By default, if you are going to, only over-sample on training set. But, the key question is do you have a ...
GooJ's user avatar
  • 435
2 votes

How to predict the quality (as a classification) of a regression?

In general this idea isn't bad BUT you need to collect new data for this to work. Modeling isn't suspect to p-hacking but this would be quite similiar. 50% of predicted outcomes not within tolerance ...
Fnguyen's user avatar
  • 1,733
2 votes

CNN model for binary classification

These two charts are showing how the accuracy and loss changed with each epoch. This model is trained for 10 epochs and the first chart is showing the training accuracy and validation accuracy from ...
sourin_karmakar's user avatar
2 votes

ROC Curve for model validation

The only ways that I can see how a ROC curve could be used for model validation is to check that it is above the $45$-degree line from $(0,0)$ to $(1,1)$. If the curve is below this, then then model ...
Dave's user avatar
  • 3,698
2 votes

ROC Curve for model validation

AUC is generally a good, relatively stable metric for evaluating a binary classifier. Looking at individual ROC curves not so much. The standard definition 'area under the ROC curve' translate that ...
Lucas Morin's user avatar
  • 2,093
2 votes

Mixed effects models for a classification task on panel data

I am actually looking for something similar, however, in my case I am looking for something to deal with multiclass labeled data. In my search, I came across GPBoost, which may just satisfy your needs....
Pim7's user avatar
  • 21
2 votes
Accepted

What does precision-recall curve and ROC curve tell us abouth threshold invariance

Does this essentially mean that an L-shaped curve means that the model performs equally well [...] for all classification thresholds No it doesn't. Let's consider a perfect predictor: all the ...
Calimo's user avatar
  • 138

Only top scored, non community-wiki answers of a minimum length are eligible