I am running some experiments in Azure using AutoML. My problem is a binary classification one, with highly imbalanced classes (basically trying to predict what factors make a deal "WON" versus "LOST", and "WON" accounts for 2% of the data).

When running the experiment, I am looking at the metrics, specifically, recall and precision, yet these figures appear (to me anyway), to be oriented towards the majority class rather than the minority class (and it's the minority class I really care about). I have shown the "Macro", "Micro" and "Weighted" Version of these metrics. To make my point let's focus on "Recall" which has values of 0.542, 0.981, 0.981 depending on which method is used (Macro, Micros, etc).

However - looking at the values on the confusion matrix, (from top left to bottom right), the numbers are 14453,15,266,25 (< which I reproduce here in case they are difficult to see on the image).

If I work out the recall for the minority class (based on [ 25 / (25+266] ], I get 0.086, which is "poor" (subjective judgment!).

Can anyone throw light on this ?, or any advice for using AutoML to help me get "better" results for my minority class?

Metrics and Confusion Matrix


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.