6
$\begingroup$

I built a CNN to learn to classify EEG data (only about 4000 training examples, 2 classes, 50-50 class balance). Each training example is 64x512, with 5 channels each

Ive tried to keep the network as simple/small as possible for testing:

  1. ConvLayer (4 filters)
  2. MaxPool
  3. Dropout 50%
  4. Fully connected (50 neurons)
  5. Dropout 50%
  6. Softmax

Im also using weight decay (L2 reg, lambda = 0.001)

The problem is no matter how I play with the filter parameters (size, stride, number) my network keeps overfitting. It fits the training data 100%, but no matter what I do I can't get the test accuracy over 65%.

Why is such a small network overfitting? I thought it was a sample size issue, but I've read a number of research papers on EEG and BCI and they occasionally have even smaller sample sizes than I do

What else can be done to regularize a CNN?

$\endgroup$
  • 1
    $\begingroup$ Pls explain why your EEG data are 2-dimensional and have 5 channels. Are they images? If so the classes they belong is associated with a certain specific pattern? $\endgroup$ – horaceT Feb 4 '17 at 20:43
3
$\begingroup$

This is because you have very little amount of data. If not enough data is provided to CNNs, they are very likely to overfit. You can do the following things in order to overcome this problem:

Data Augmentation : It is a technique to create new examples from the training examples by doing some preprocessing on them e.g. Rotation, Scaling, etc.

Rigorous Dropout : Dropout is a very powerful technique to control overfittting. You can try using more rigorous dropout in your architecture

$\endgroup$
  • 1
    $\begingroup$ Could you define what you mean by rigorous dropout in this case? How could I make it more rigorous? $\endgroup$ – Simon Dec 6 '16 at 9:45
  • $\begingroup$ You can increase the dropout to 75% in one of the layers and see what happens but data augmentation is much more important to solve your problem. $\endgroup$ – Nain Dec 6 '16 at 9:46
  • $\begingroup$ I'd add: Aggressively downsample the input data. E.g. from 64 x 512 to 16 x 128. $\endgroup$ – Neil Slater Dec 6 '16 at 11:37
  • 1
    $\begingroup$ Additionally, I would add that he should check if he can make the network not to overfit, e.g. by having not 50 but only 1 neuron. Or 2. Just to see if that is really the problem or something else. $\endgroup$ – Martin Thoma Dec 16 '16 at 23:25
1
$\begingroup$

Have you tried early stopping? Hold out 20% of the data and stop training when the error on the hold-out set starts to increase. If you can't get high enough accuracy, increase the size of your model until you do.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.