426
votes
Micro Average vs Macro average Performance in a Multiclass classification setting
Micro- and macro-averages (for whatever metric) will compute slightly different things, and thus their interpretation differs. A macro-average will compute the metric independently for each class and ...
- 4,369
40
votes
Micro Average vs Macro average Performance in a Multiclass classification setting
This is the Original Post.
In Micro-average method, you sum up the individual true positives, false positives, and false negatives of the system for different sets and the apply them to get the ...
27
votes
Micro Average vs Macro average Performance in a Multiclass classification setting
In a multi-class setting micro-averaged precision and recall are always the same.
$$
P = \frac{\sum_c TP_c}{\sum_c TP_c + \sum_c FP_c}\\
R = \frac{\sum_c TP_c}{\sum_c TP_c + \sum_c FN_c}
$$
where c ...
- 430
26
votes
Accepted
Unbalanced multiclass data with XGBoost
scale_pos_weight is used for binary classification as you stated. It is a more generalized solution to handle imbalanced classes. A good approach when assigning a ...
- 411
18
votes
Unbalanced multiclass data with XGBoost
This answer by @KeremT is correct. I provide an example for those who still have problems with the exact implementation.
weight parameter in XGBoost is per ...
- 8,629
16
votes
Unbalanced multiclass data with XGBoost
For sklearn version < 0.19
Just assign each entry of your train data its class weight. First get the class weights with class_weight.compute_class_weight of ...
- 271
16
votes
Accepted
Which comes first? Multiple Imputation, Splitting into train/test, or Standardization/Normalization
Always split before you do any data pre-processing. Performing pre-processing before splitting will mean that information from your test set will be present during training, causing a data leak.
...
- 3,868
14
votes
weighted cross entropy for imbalanced dataset - multiclass classification
If you are looking for just an alternative loss function:
Focal Loss has been shown on imagenet to help with this problem indeed.
Focal loss adds a modulating factor to cross entropy loss ensuring ...
- 141
13
votes
What is the best method for classification of time series data? Should I use LSTM or a different method?
I would not use the word "best" but LSTM-RNN are very powerful when dealing with timeseries, simply because they can store information about previous values and exploit the time dependencies ...
- 3,740
12
votes
Accepted
Why class weight is outperforming oversampling?
You should not expect class_weight parameters and SMOTE to give the exact same results because they are different methods.
Class weights directly modify the loss function by giving more (or less) ...
- 1,834
10
votes
Keras Multiple “Softmax” in last layer possible?
I would use the functional interface.
Something like this:
...
- 17.6k
9
votes
How does Sigmoid activation work in multi-class classification problems
If your task is a kind of classification that the labels are mutually exclusive, each input just has one label, you have to use Softmax. If the inputs of your ...
- 13.4k
9
votes
Deep network not able to learn imbalanced data beyond the dominant class
1) A five-layer neural network is one heck of a complex model for a data set with less than 1 million points. (I’m trying to find a good link for this, but the intuition is that your choice of model ...
- 191
9
votes
SGDClassifier: Online Learning/partial_fit with a previously unknown label
It sounds like you don't want to start retraining the model every time a new label category appears. The easiest way to retain maximal information of past data would be train one classifier per ...
oW_♦
- 5,995
8
votes
Accepted
Multi-class classification v.s. Binary classification
The greater the number of output nodes the higher complexity you will add to your model. This means that given a fixed amount of data, a greater number of output nodes will lead to poorer results. I ...
- 8,518
7
votes
Multi-class neural net always predicting 1 class after optimization
It could be a bug in your code, problems with your training set (maybe you don't have the file format quite right), or some other implementation issue.
Are you sure you want to use a sigmoid ...
- 3,028
7
votes
Accepted
Multiclass classification with Neural Networks
Indeed, this is the standard interpretation of continuous classifier outputs, not only for neural networks, but for the more general case called Softmax Regression.
Thus, provided that you have used ...
- 1,800
6
votes
Unbalanced multiclass data with XGBoost
Everyone stumbles upon this question when dealing with unbalanced multiclass classification problem using XGBoost in R. I did too!
I was looking for an example to better understand how to apply it. ...
- 201
6
votes
Accepted
How can I have an "undefined" category in multi-class classification
Absolutely you can create an approach that forces high-precision class tagging algorithm (at the natural cost of recall). What's more- you can do this with (at least) any method that provides a ...
- 1,505
6
votes
Accepted
Imbalanced data causing mis-classification on multiclass dataset
Nice question!
Some Remarks
For imbalanced data you have different approaches. Most well-established one is resampling (Oversampling small classes /underssampling large classes). The other one is to ...
- 6,332
6
votes
Products classification by name
If you have enough data and reasonable number of classes, you can definitely train your model. The grouping of words that you have done is similar to an approach called bag-of-words model. You can use ...
- 781
6
votes
Accepted
F1_score(average='micro') is equal to calculating accuracy for multiclasification
In classification tasks for which every test case is guaranteed to be assigned to exactly one class, micro-F is equivalent to accuracy.
The above answer is from:
https://stackoverflow.com/questions/...
5
votes
Accepted
Appropriate algorithm for string (not document) classification?
You might find it useful to treat n-grams of characters as your feature space. Then you could represent a string as a bag of substrings. With N = 4 or greater, you ...
- 3,027
5
votes
Text-Classification-Problem: Is Word2Vec/NN the best approach?
1) Max-Entropy(Logistic Regression) on TFIDF vectors is a good starting point for many NLP classification task.
2) Word2vec is definitely something worth trying and comparing to model 1. I would ...
- 51
5
votes
Signs there are too many class labels
A class taxonomy should:
Serve the business needs
Be learnable
There is a potential tradeoff here. The more exact and specific the taxonomy, the more you'll know about the entities and you'll be ...
- 2,593
5
votes
Micro Average vs Macro average Performance in a Multiclass classification setting
That's how it should be. I had the same result for my research. It seemed weird at first. But precision and recall should be the same while micro-averaging the result of multi-class single-label ...
5
votes
Multi-class neural net always predicting 1 class after optimization
You learn a lot by comparing to a naive model. A naive model is one without any features. As a default, it will always predict the most likely Target. Note that this is exactly what your model is ...
- 255
5
votes
Keras Multiple “Softmax” in last layer possible?
It is possible just implement your own softmax function. You can split a tensor to parts, then compute softmax separately per part and concatenate tensor parts:
...
- 198
5
votes
Training data for multi-category classification algorithm
No, it is perfectly possible to train on multiple categories. What you need, though, is an exhaustive list of these categories (in supervised learning, that is).
Suppose you are trying to associate ...
- 1,606
5
votes
How does Sigmoid activation work in multi-class classification problems
softmax() will give you the probability distribution which means all output will sum to 1. While, sigmoid() will make sure the output value of neuron is between 0 to 1.
In case of digit ...
- 588
Only top scored, non community-wiki answers of a minimum length are eligible