Skip to main content
487 votes

Micro Average vs Macro average Performance in a Multiclass classification setting

Micro- and macro-averages (for whatever metric) will compute slightly different things, and thus their interpretation differs. A macro-average will compute the metric independently for each class and ...
pythiest's user avatar
  • 4,979
46 votes

Micro Average vs Macro average Performance in a Multiclass classification setting

This is the Original Post. In Micro-average method, you sum up the individual true positives, false positives, and false negatives of the system for different sets and the apply them to get the ...
Rahul Reddy Vemireddy's user avatar
30 votes
Accepted

Unbalanced multiclass data with XGBoost

scale_pos_weight is used for binary classification as you stated. It is a more generalized solution to handle imbalanced classes. A good approach when assigning a ...
Kerem T's user avatar
  • 461
28 votes

Micro Average vs Macro average Performance in a Multiclass classification setting

In a multi-class setting micro-averaged precision and recall are always the same. $$ P = \frac{\sum_c TP_c}{\sum_c TP_c + \sum_c FP_c}\\ R = \frac{\sum_c TP_c}{\sum_c TP_c + \sum_c FN_c} $$ where c ...
David Makovoz's user avatar
21 votes

Unbalanced multiclass data with XGBoost

This answer by @KeremT is correct. I provide an example for those who still have problems with the exact implementation. weight parameter in XGBoost is per ...
Esmailian's user avatar
  • 9,342
18 votes

Unbalanced multiclass data with XGBoost

For sklearn version < 0.19 Just assign each entry of your train data its class weight. First get the class weights with class_weight.compute_class_weight of ...
Firas Omrane's user avatar
18 votes
Accepted

Which comes first? Multiple Imputation, Splitting into train/test, or Standardization/Normalization

Always split before you do any data pre-processing. Performing pre-processing before splitting will mean that information from your test set will be present during training, causing a data leak. ...
Simon Larsson's user avatar
16 votes
Accepted

Why class weight is outperforming oversampling?

You should not expect class_weight parameters and SMOTE to give the exact same results because they are different methods. Class weights directly modify the loss function by giving more (or less) ...
aranglol's user avatar
  • 2,196
14 votes

weighted cross entropy for imbalanced dataset - multiclass classification

If you are looking for just an alternative loss function: Focal Loss has been shown on imagenet to help with this problem indeed. Focal loss adds a modulating factor to cross entropy loss ensuring ...
Vibhu Jawa's user avatar
13 votes

What is the best method for classification of time series data? Should I use LSTM or a different method?

I would not use the word "best" but LSTM-RNN are very powerful when dealing with timeseries, simply because they can store information about previous values and exploit the time dependencies ...
pcko1's user avatar
  • 3,950
12 votes

How does Sigmoid activation work in multi-class classification problems

If your task is a kind of classification that the labels are mutually exclusive, each input just has one label, you have to use Softmax. If the inputs of your ...
Green Falcon's user avatar
  • 14.1k
10 votes

Keras Multiple “Softmax” in last layer possible?

I would use the functional interface. Something like this: ...
Martin Thoma's user avatar
9 votes

Deep network not able to learn imbalanced data beyond the dominant class

1) A five-layer neural network is one heck of a complex model for a data set with less than 1 million points. (I’m trying to find a good link for this, but the intuition is that your choice of model ...
DGrady's user avatar
  • 191
9 votes

SGDClassifier: Online Learning/partial_fit with a previously unknown label

It sounds like you don't want to start retraining the model every time a new label category appears. The easiest way to retain maximal information of past data would be train one classifier per ...
oW_'s user avatar
  • 6,367
8 votes
Accepted

Multi-class classification v.s. Binary classification

The greater the number of output nodes the higher complexity you will add to your model. This means that given a fixed amount of data, a greater number of output nodes will lead to poorer results. I ...
JahKnows's user avatar
  • 8,946
8 votes
Accepted

Micro Average vs Macro Average for Class Imbalance

The question is actually about understanding what it means to "take imbalance into account": Micro-average "takes imbalance into account" in the sense that the resulting ...
Erwan's user avatar
  • 25.5k
7 votes

Micro Average vs Macro average Performance in a Multiclass classification setting

Assume that we are classifying an email into one of the three groups: urgent, normal and spam. We compare the predicts with the ground truth labels, then we get the following confusion matrix and the ...
Lerner Zhang's user avatar
7 votes

Multi-class neural net always predicting 1 class after optimization

It could be a bug in your code, problems with your training set (maybe you don't have the file format quite right), or some other implementation issue. Are you sure you want to use a sigmoid ...
D.W.'s user avatar
  • 3,381
7 votes

How does Sigmoid activation work in multi-class classification problems

softmax() will give you the probability distribution which means all output will sum to 1. While, sigmoid() will make sure the output value of neuron is between 0 to 1. In case of digit ...
Preet's user avatar
  • 638
7 votes
Accepted

True positives and true negatives, F1 score: multi class classification

In a multiclass problem there is one score for each class, counting any other class as a negative. For example for class 1: TP instances are gold standard class 1 predicted as class 1 FN instances ...
Erwan's user avatar
  • 25.5k
7 votes
Accepted

F1_score(average='micro') is equal to calculating accuracy for multiclasification

In classification tasks for which every test case is guaranteed to be assigned to exactly one class, micro-F is equivalent to accuracy. The above answer is from: https://stackoverflow.com/questions/...
Saandeep Sreerambatla's user avatar
6 votes

Micro Average vs Macro average Performance in a Multiclass classification setting

That's how it should be. I had the same result for my research. It seemed weird at first. But precision and recall should be the same while micro-averaging the result of multi-class single-label ...
Saghan Mudbhari's user avatar
6 votes

Unbalanced multiclass data with XGBoost

Everyone stumbles upon this question when dealing with unbalanced multiclass classification problem using XGBoost in R. I did too! I was looking for an example to better understand how to apply it. ...
Krithi07's user avatar
  • 211
6 votes
Accepted

How can I have an "undefined" category in multi-class classification

Absolutely you can create an approach that forces high-precision class tagging algorithm (at the natural cost of recall). What's more- you can do this with (at least) any method that provides a ...
Thomas Cleberg's user avatar
6 votes
Accepted

Multiclass classification with Neural Networks

Indeed, this is the standard interpretation of continuous classifier outputs, not only for neural networks, but for the more general case called Softmax Regression. Thus, provided that you have used ...
desertnaut's user avatar
  • 2,018
6 votes
Accepted

Imbalanced data causing mis-classification on multiclass dataset

Nice question! Some Remarks For imbalanced data you have different approaches. Most well-established one is resampling (Oversampling small classes /underssampling large classes). The other one is to ...
Kasra Manshaei's user avatar
6 votes

Products classification by name

If you have enough data and reasonable number of classes, you can definitely train your model. The grouping of words that you have done is similar to an approach called bag-of-words model. You can use ...
Gyan Ranjan's user avatar
6 votes
Accepted

Products classification by name

I think, and have done similar problem too, that this problem can be solved in this way: 1. Generate NGrams 2. Create 1 hot encoding matrix 3. Pass to Naive Bayes or Random forest It would ...
Sandeep Bhutani's user avatar
5 votes

Signs there are too many class labels

A class taxonomy should: Serve the business needs Be learnable There is a potential tradeoff here. The more exact and specific the taxonomy, the more you'll know about the entities and you'll be ...
DaL's user avatar
  • 2,643
5 votes

Multi-class neural net always predicting 1 class after optimization

You learn a lot by comparing to a naive model. A naive model is one without any features. As a default, it will always predict the most likely Target. Note that this is exactly what your model is ...
Paul's user avatar
  • 255

Only top scored, non community-wiki answers of a minimum length are eligible