I am doing multilabel news classification in python language.The dataset I have has two files. First CSV contains articles at each row. Second CSV contains corresponding labels to each article. Here is the snapshot of label file

enter image description here

Here is the dropbox link....https://www.dropbox.com/s/7huzh41je735oqn/labelset.csv?dl=0

  1. Is the dataset imbalanced one?
  2. How do I distribute this dataset properly into training set, validation and test set?

Note: By properly I mean, Can this unbalanced dataset be split into proper proportions in training,validation and testing set?

  • $\begingroup$ What models are you working with? $\endgroup$ – Leevo Jun 12 '19 at 22:21
  • $\begingroup$ Sorry I did not understand your question....I am working with keras right now. Is that what you are asking? $\endgroup$ – Annonymous programmer Jun 12 '19 at 22:35
  • 1
    $\begingroup$ Ok, so you are working with Neural Networks. $\endgroup$ – Leevo Jun 13 '19 at 7:04
  • $\begingroup$ Now I understand what you mean.....:D $\endgroup$ – Annonymous programmer Jun 13 '19 at 10:05

I'll go through your questions one by one:

Is the dataset imbalanced one?

The distinction between a balanced and an imbalanced dataset is a matter of degree. If you think that the difference in number of observations for each class is too big, I suggest you to train your model using Mini-Batch Gradient Descent, and build any mini batch in a way that observations have the same frequency. This would attribute greater weights to the less frequent observations.

How do I distribute this dataset properly into training set, validation and test set?

Make sure that all the classes are evenly distributed among the sets.

| improve this answer | |

Yes, the dataset you have been working with is an imbalanced dataset but this problem could easily be tackled if rather than predicting the outcome directly, you will tackle this problem by top 3 predictions. Over the top 3 predictions, you can also add one more check for the threshold to validate the predictions and finalize the prediction.

Apart from this, I would suggest you to manually split the dataset into training and validation set in order to maintain a consistency in the data and help tackle co-variate shift problem.

| improve this answer | |
  • $\begingroup$ You mean I need to see the results for only top 3 predictions and validate it? $\endgroup$ – Annonymous programmer Jun 13 '19 at 10:05
  • $\begingroup$ Yes, this should solve the problem. $\endgroup$ – thanatoz Jun 13 '19 at 10:20
  • $\begingroup$ I am confused....By top 3 predictions what do you mean? $\endgroup$ – Annonymous programmer Jun 13 '19 at 11:50
  • $\begingroup$ By top 3 predictions I mean the classes predicted with top confidences. The confidence of each prediction can be obtained by applying the softmax layer in the last layer. $\endgroup$ – thanatoz Jun 13 '19 at 11:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.