493
votes
Micro Average vs Macro average Performance in a Multiclass classification setting
Micro- and macro-averages (for whatever metric) will compute slightly different things, and thus their interpretation differs. A macro-average will compute the metric independently for each class and ...
- 5,039
46
votes
Micro Average vs Macro average Performance in a Multiclass classification setting
This is the Original Post.
In Micro-average method, you sum up the individual true positives, false positives, and false negatives of the system for different sets and the apply them to get the ...
31
votes
Accepted
Unbalanced multiclass data with XGBoost
scale_pos_weight is used for binary classification as you stated. It is a more generalized solution to handle imbalanced classes. A good approach when assigning a ...
- 471
28
votes
Micro Average vs Macro average Performance in a Multiclass classification setting
In a multi-class setting micro-averaged precision and recall are always the same.
$$
P = \frac{\sum_c TP_c}{\sum_c TP_c + \sum_c FP_c}\\
R = \frac{\sum_c TP_c}{\sum_c TP_c + \sum_c FN_c}
$$
where c ...
- 470
21
votes
Unbalanced multiclass data with XGBoost
This answer by @KeremT is correct. I provide an example for those who still have problems with the exact implementation.
weight parameter in XGBoost is per ...
- 9,402
18
votes
Unbalanced multiclass data with XGBoost
For sklearn version < 0.19
Just assign each entry of your train data its class weight. First get the class weights with class_weight.compute_class_weight of ...
- 291
18
votes
Accepted
Which comes first? Multiple Imputation, Splitting into train/test, or Standardization/Normalization
Always split before you do any data pre-processing. Performing pre-processing before splitting will mean that information from your test set will be present during training, causing a data leak.
...
- 4,243
16
votes
Accepted
Why class weight is outperforming oversampling?
You should not expect class_weight parameters and SMOTE to give the exact same results because they are different methods.
Class weights directly modify the loss function by giving more (or less) ...
- 2,206
15
votes
weighted cross entropy for imbalanced dataset - multiclass classification
If you are looking for just an alternative loss function:
Focal Loss has been shown on imagenet to help with this problem indeed.
Focal loss adds a modulating factor to cross entropy loss ensuring ...
- 151
13
votes
What is the best method for classification of time series data? Should I use LSTM or a different method?
I would not use the word "best" but LSTM-RNN are very powerful when dealing with timeseries, simply because they can store information about previous values and exploit the time dependencies ...
- 3,970
12
votes
How does Sigmoid activation work in multi-class classification problems
If your task is a kind of classification that the labels are mutually exclusive, each input just has one label, you have to use Softmax. If the inputs of your ...
- 14.1k
10
votes
Keras Multiple “Softmax” in last layer possible?
I would use the functional interface.
Something like this:
...
- 19.1k
9
votes
Deep network not able to learn imbalanced data beyond the dominant class
1) A five-layer neural network is one heck of a complex model for a data set with less than 1 million points. (I’m trying to find a good link for this, but the intuition is that your choice of model ...
- 191
9
votes
SGDClassifier: Online Learning/partial_fit with a previously unknown label
It sounds like you don't want to start retraining the model every time a new label category appears. The easiest way to retain maximal information of past data would be train one classifier per ...
- 6,422
8
votes
Micro Average vs Macro average Performance in a Multiclass classification setting
Assume that we are classifying an email into one of the three groups: urgent, normal and spam. We compare the predicts with the ground truth labels, then we get the following confusion matrix and the ...
- 526
8
votes
Accepted
Multi-class classification v.s. Binary classification
The greater the number of output nodes the higher complexity you will add to your model. This means that given a fixed amount of data, a greater number of output nodes will lead to poorer results. I ...
- 8,986
8
votes
Accepted
Micro Average vs Macro Average for Class Imbalance
The question is actually about understanding what it means to "take imbalance into account":
Micro-average "takes imbalance into account" in the sense that the resulting ...
- 25.7k
7
votes
Multi-class neural net always predicting 1 class after optimization
It could be a bug in your code, problems with your training set (maybe you don't have the file format quite right), or some other implementation issue.
Are you sure you want to use a sigmoid ...
- 3,401
7
votes
How does Sigmoid activation work in multi-class classification problems
softmax() will give you the probability distribution which means all output will sum to 1. While, sigmoid() will make sure the output value of neuron is between 0 to 1.
In case of digit ...
- 638
7
votes
Accepted
True positives and true negatives, F1 score: multi class classification
In a multiclass problem there is one score for each class, counting any other class as a negative.
For example for class 1:
TP instances are gold standard class 1 predicted as class 1
FN instances ...
- 25.7k
7
votes
Accepted
F1_score(average='micro') is equal to calculating accuracy for multiclasification
In classification tasks for which every test case is guaranteed to be assigned to exactly one class, micro-F is equivalent to accuracy.
The above answer is from:
https://stackoverflow.com/questions/...
6
votes
Micro Average vs Macro average Performance in a Multiclass classification setting
That's how it should be. I had the same result for my research. It seemed weird at first. But precision and recall should be the same while micro-averaging the result of multi-class single-label ...
6
votes
Unbalanced multiclass data with XGBoost
Everyone stumbles upon this question when dealing with unbalanced multiclass classification problem using XGBoost in R. I did too!
I was looking for an example to better understand how to apply it. ...
- 211
6
votes
Accepted
How can I have an "undefined" category in multi-class classification
Absolutely you can create an approach that forces high-precision class tagging algorithm (at the natural cost of recall). What's more- you can do this with (at least) any method that provides a ...
- 1,525
6
votes
Accepted
Multiclass classification with Neural Networks
Indeed, this is the standard interpretation of continuous classifier outputs, not only for neural networks, but for the more general case called Softmax Regression.
Thus, provided that you have used ...
- 2,017
6
votes
Accepted
Imbalanced data causing mis-classification on multiclass dataset
Nice question!
Some Remarks
For imbalanced data you have different approaches. Most well-established one is resampling (Oversampling small classes /underssampling large classes). The other one is to ...
- 6,622
6
votes
Products classification by name
If you have enough data and reasonable number of classes, you can definitely train your model. The grouping of words that you have done is similar to an approach called bag-of-words model. You can use ...
- 811
6
votes
Accepted
Products classification by name
I think, and have done similar problem too, that this problem can be solved in this way:
1. Generate NGrams
2. Create 1 hot encoding matrix
3. Pass to Naive Bayes or Random forest
It would ...
- 894
5
votes
Signs there are too many class labels
A class taxonomy should:
Serve the business needs
Be learnable
There is a potential tradeoff here. The more exact and specific the taxonomy, the more you'll know about the entities and you'll be ...
- 2,663
5
votes
Multi-class neural net always predicting 1 class after optimization
You learn a lot by comparing to a naive model. A naive model is one without any features. As a default, it will always predict the most likely Target. Note that this is exactly what your model is ...
- 255
Only top scored, non community-wiki answers of a minimum length are eligible