426 votes

Micro Average vs Macro average Performance in a Multiclass classification setting

Micro- and macro-averages (for whatever metric) will compute slightly different things, and thus their interpretation differs. A macro-average will compute the metric independently for each class and ...
user avatar
  • 4,369
40 votes

Micro Average vs Macro average Performance in a Multiclass classification setting

This is the Original Post. In Micro-average method, you sum up the individual true positives, false positives, and false negatives of the system for different sets and the apply them to get the ...
user avatar
27 votes

Micro Average vs Macro average Performance in a Multiclass classification setting

In a multi-class setting micro-averaged precision and recall are always the same. $$ P = \frac{\sum_c TP_c}{\sum_c TP_c + \sum_c FP_c}\\ R = \frac{\sum_c TP_c}{\sum_c TP_c + \sum_c FN_c} $$ where c ...
user avatar
26 votes
Accepted

Unbalanced multiclass data with XGBoost

scale_pos_weight is used for binary classification as you stated. It is a more generalized solution to handle imbalanced classes. A good approach when assigning a ...
user avatar
  • 411
18 votes

Unbalanced multiclass data with XGBoost

This answer by @KeremT is correct. I provide an example for those who still have problems with the exact implementation. weight parameter in XGBoost is per ...
user avatar
  • 8,629
16 votes

Unbalanced multiclass data with XGBoost

For sklearn version < 0.19 Just assign each entry of your train data its class weight. First get the class weights with class_weight.compute_class_weight of ...
user avatar
16 votes
Accepted

Which comes first? Multiple Imputation, Splitting into train/test, or Standardization/Normalization

Always split before you do any data pre-processing. Performing pre-processing before splitting will mean that information from your test set will be present during training, causing a data leak. ...
user avatar
14 votes

weighted cross entropy for imbalanced dataset - multiclass classification

If you are looking for just an alternative loss function: Focal Loss has been shown on imagenet to help with this problem indeed. Focal loss adds a modulating factor to cross entropy loss ensuring ...
user avatar
13 votes

What is the best method for classification of time series data? Should I use LSTM or a different method?

I would not use the word "best" but LSTM-RNN are very powerful when dealing with timeseries, simply because they can store information about previous values and exploit the time dependencies ...
user avatar
  • 3,740
12 votes
Accepted

Why class weight is outperforming oversampling?

You should not expect class_weight parameters and SMOTE to give the exact same results because they are different methods. Class weights directly modify the loss function by giving more (or less) ...
user avatar
  • 1,834
10 votes

Keras Multiple “Softmax” in last layer possible?

I would use the functional interface. Something like this: ...
user avatar
  • 17.6k
9 votes

How does Sigmoid activation work in multi-class classification problems

If your task is a kind of classification that the labels are mutually exclusive, each input just has one label, you have to use Softmax. If the inputs of your ...
user avatar
  • 13.4k
9 votes

Deep network not able to learn imbalanced data beyond the dominant class

1) A five-layer neural network is one heck of a complex model for a data set with less than 1 million points. (I’m trying to find a good link for this, but the intuition is that your choice of model ...
user avatar
  • 191
9 votes

SGDClassifier: Online Learning/partial_fit with a previously unknown label

It sounds like you don't want to start retraining the model every time a new label category appears. The easiest way to retain maximal information of past data would be train one classifier per ...
user avatar
  • 5,995
8 votes
Accepted

Multi-class classification v.s. Binary classification

The greater the number of output nodes the higher complexity you will add to your model. This means that given a fixed amount of data, a greater number of output nodes will lead to poorer results. I ...
user avatar
  • 8,518
7 votes

Multi-class neural net always predicting 1 class after optimization

It could be a bug in your code, problems with your training set (maybe you don't have the file format quite right), or some other implementation issue. Are you sure you want to use a sigmoid ...
user avatar
  • 3,028
7 votes
Accepted

Multiclass classification with Neural Networks

Indeed, this is the standard interpretation of continuous classifier outputs, not only for neural networks, but for the more general case called Softmax Regression. Thus, provided that you have used ...
user avatar
  • 1,800
6 votes

Unbalanced multiclass data with XGBoost

Everyone stumbles upon this question when dealing with unbalanced multiclass classification problem using XGBoost in R. I did too! I was looking for an example to better understand how to apply it. ...
user avatar
  • 201
6 votes
Accepted

How can I have an "undefined" category in multi-class classification

Absolutely you can create an approach that forces high-precision class tagging algorithm (at the natural cost of recall). What's more- you can do this with (at least) any method that provides a ...
user avatar
6 votes
Accepted

Imbalanced data causing mis-classification on multiclass dataset

Nice question! Some Remarks For imbalanced data you have different approaches. Most well-established one is resampling (Oversampling small classes /underssampling large classes). The other one is to ...
user avatar
6 votes

Products classification by name

If you have enough data and reasonable number of classes, you can definitely train your model. The grouping of words that you have done is similar to an approach called bag-of-words model. You can use ...
user avatar
6 votes
Accepted

F1_score(average='micro') is equal to calculating accuracy for multiclasification

In classification tasks for which every test case is guaranteed to be assigned to exactly one class, micro-F is equivalent to accuracy. The above answer is from: https://stackoverflow.com/questions/...
user avatar
5 votes
Accepted

Appropriate algorithm for string (not document) classification?

You might find it useful to treat n-grams of characters as your feature space. Then you could represent a string as a bag of substrings. With N = 4 or greater, you ...
user avatar
  • 3,027
5 votes

Text-Classification-Problem: Is Word2Vec/NN the best approach?

1) Max-Entropy(Logistic Regression) on TFIDF vectors is a good starting point for many NLP classification task. 2) Word2vec is definitely something worth trying and comparing to model 1. I would ...
user avatar
  • 51
5 votes

Signs there are too many class labels

A class taxonomy should: Serve the business needs Be learnable There is a potential tradeoff here. The more exact and specific the taxonomy, the more you'll know about the entities and you'll be ...
user avatar
  • 2,593
5 votes

Micro Average vs Macro average Performance in a Multiclass classification setting

That's how it should be. I had the same result for my research. It seemed weird at first. But precision and recall should be the same while micro-averaging the result of multi-class single-label ...
user avatar
5 votes

Multi-class neural net always predicting 1 class after optimization

You learn a lot by comparing to a naive model. A naive model is one without any features. As a default, it will always predict the most likely Target. Note that this is exactly what your model is ...
user avatar
  • 255
5 votes

Keras Multiple “Softmax” in last layer possible?

It is possible just implement your own softmax function. You can split a tensor to parts, then compute softmax separately per part and concatenate tensor parts: ...
user avatar
  • 198
5 votes

Training data for multi-category classification algorithm

No, it is perfectly possible to train on multiple categories. What you need, though, is an exhaustive list of these categories (in supervised learning, that is). Suppose you are trying to associate ...
user avatar
5 votes

How does Sigmoid activation work in multi-class classification problems

softmax() will give you the probability distribution which means all output will sum to 1. While, sigmoid() will make sure the output value of neuron is between 0 to 1. In case of digit ...
user avatar
  • 588

Only top scored, non community-wiki answers of a minimum length are eligible