7
$\begingroup$

I'm currently facing a Machine Learning problem and I've reached a point where I need some help to proceed.

I have various time series of positional (x, y, z) data tracked by sensors. I've developed some more features. For example, I rasterized the whole 3D space and calculated a cell_x, cell_y and cell_z for every time step. The time series itself have variable lengths.

My goal is to build a model which classifies every time step with the labels 0 or 1 (binary classification based on past and future values). Therefore I have a lot of training time series where the labels are already set.

One thing which could be very problematic is that there are very few 1's labels in the data (for example only 3 of 800 samples are labeled with 1).

It would be great if someone can help me in the right direction because there are too many possible problems:

  • Wrong hyperparameters
  • Incorrect model
  • Too few 1's labels, but I think that's not a big problem because I only need the model to suggests the right time steps. So I would only use the peaks of the output.
  • Bad or too less training data
  • Bad features

I appreciate any help and tips.

$\endgroup$
7
$\begingroup$

You are facing a very common problem: handling imbalanced data. For neural networks, typical procedures are:

  1. Having the proper metrics: global accuracy should not be used.
  2. Oversampling the minority class: randomly generate replicas of the minority class until the imbalance disappears.You can also perform data augmentation on the minority class. Synthetic data can be generated from the feature space, using SMOTE algorithm, but I don't know how it applies to neural networks.
  3. Under-sampling the majority class: randomly remove instances of the majority class. It can deteriorate the performance on the majority class
  4. Include class-weights in the loss function: the idea is to penalize the misclassification of the minority class. The weights are usually inversely proportional to the occurrence frequency of each class.
  5. Using different learning rates per class: you can use a bigger learning rate for the majority class, thus the net stops learning earlier from the majority class than from the minority class

I'd recommend a combination of 1 (always), 2 and 4. For a higher insight in this topic, which is of very importance, I recommend reading:

  1. https://dl.acm.org/citation.cfm?id=1592322
  2. https://arxiv.org/pdf/1710.05381.pdf
| improve this answer | |
$\endgroup$
  • $\begingroup$ Do you have an example for 1.? $\endgroup$ – Chryb May 13 '18 at 12:08
  • 1
    $\begingroup$ You can use metrics from the confusion matrix. I usually take a look at precision and recall per class, you can also take a llok at F1 score. For segmentation or object detection, another well known metric is the Intersection Over Union (IoU) $\endgroup$ – ignatius May 14 '18 at 9:59
  • $\begingroup$ Did you know how to apply the confusion matrix to keras? $\endgroup$ – Chryb May 14 '18 at 12:45
  • $\begingroup$ Ok, I found this: stackoverflow.com/questions/43547402/… $\endgroup$ – Chryb May 14 '18 at 16:33
  • $\begingroup$ Did you ever come up with a solution? I have the exact problem and I am looking for a MWE. $\endgroup$ – John Stud Apr 29 at 17:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.