0
$\begingroup$

Now I'm using Keras to implement a multi-label classification model. Specifically, I want to classify who present in an audio clip (maximal 8 people). The label of data has 8-bit, for example, [0,1,0,0,1,0,1,1]. It means totally the data should have 2^8=256 combinations. Now I only collected part of the data (3700 samples, but only with 20 labels) for the model training. Although the model has a good performance in seen data, it performs badly for the data with unseen labels(data with other 236 labels). I wonder how I can improve the model performance? or I have to train this model with as much as data with different labels? I think it will cause a combinatorial explosion for the data collecting workload.

$\endgroup$
3
  • $\begingroup$ It would be better to fully explain the task (i.e. what you're classifying) as the amount of data required doesn't only depend on the label space. Also, do you mean you only have 20 data samples, or you have more but amongst them only 20 of the possible 256 labels occur? $\endgroup$
    – Carl
    Jun 16 '21 at 15:51
  • $\begingroup$ Hi Carl. I just edited my task. I want to classify who present in an audio clip (maximal 8 people). The 20 means I collected 3700 samples but only with 20 labels among 256. $\endgroup$
    – Qiang
    Jun 17 '21 at 2:17
  • $\begingroup$ Thanks, much clearer. $\endgroup$
    – Carl
    Jun 18 '21 at 11:33
0
$\begingroup$

General rule: use all the data you can get, keeping aside a test set and validation set.

It is unlikely there is a combinatoric issue as an 8 dimensional output is not large in the context of machine learning. Also, not all of the $2^8$ possible labels may occur, due to correlation or other inter-relationships between label dimensions (depending on the data). For example, in the extreme, all digits might always be the same so only two labels ever occur: (0,0,0, ...) and (1,1,1, ...) meaning there is effectively only 1 classification task rather than 8. In your case, maybe certain people always speak at the same time. If any combination is possible then it really is effectively 8 independent tasks, but that is not necessarily a problem.

The data requirement here is more about you learning a pattern in very-high dimensional data (sound clips) from relatively few samples (I don't know how many dimension each clip has but presumably mroe than the number of samples you have?). That means there are many possible functions that map x to y for the training data, many of which may not capture the latent structure needed to generalise (i.e. they are finding spurious correlations). In short, you are probably over-fitting. If so, remedies are:

  1. get more labelled data if possible;
  2. regularise, e.g. $\ell_2$ to keep weights small, $\ell_1$ to make them sparse, dropout; and
  3. if you can get unlabelled sound clips use semi-supervised learning.
$\endgroup$
2
  • $\begingroup$ Thanks. I will get more data and have a try. I guess there is some kind of independence among these labels (eg., two persons speak at the same time). I have revised my question. $\endgroup$
    – Qiang
    Jun 17 '21 at 2:31
  • $\begingroup$ Thanks for your suggestion! $\endgroup$
    – Qiang
    Jun 19 '21 at 12:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.