I built a CNN to learn to classify EEG data (only about 4000 training examples, 2 classes, 50-50 class balance). Each training example is 64x512, with 5 channels each

Ive tried to keep the network as simple/small as possible for testing:

  1. ConvLayer (4 filters)
  2. MaxPool
  3. Dropout 50%
  4. Fully connected (50 neurons)
  5. Dropout 50%
  6. Softmax

Im also using weight decay (L2 reg, lambda = 0.001)

The problem is no matter how I play with the filter parameters (size, stride, number) my network keeps overfitting. It fits the training data 100%, but no matter what I do I can't get the test accuracy over 65%.

Why is such a small network overfitting? I thought it was a sample size issue, but I've read a number of research papers on EEG and BCI and they occasionally have even smaller sample sizes than I do

What else can be done to regularize a CNN?

  • 1
    $\begingroup$ Pls explain why your EEG data are 2-dimensional and have 5 channels. Are they images? If so the classes they belong is associated with a certain specific pattern? $\endgroup$
    – horaceT
    Commented Feb 4, 2017 at 20:43

3 Answers 3


This is because you have very little amount of data. If not enough data is provided to CNNs, they are very likely to overfit. You can do the following things in order to overcome this problem:

Data Augmentation : It is a technique to create new examples from the training examples by doing some preprocessing on them e.g. Rotation, Scaling, etc.

Rigorous Dropout : Dropout is a very powerful technique to control overfittting. You can try using more rigorous dropout in your architecture

  • 1
    $\begingroup$ Could you define what you mean by rigorous dropout in this case? How could I make it more rigorous? $\endgroup$
    – Simon
    Commented Dec 6, 2016 at 9:45
  • $\begingroup$ You can increase the dropout to 75% in one of the layers and see what happens but data augmentation is much more important to solve your problem. $\endgroup$
    – enterML
    Commented Dec 6, 2016 at 9:46
  • $\begingroup$ I'd add: Aggressively downsample the input data. E.g. from 64 x 512 to 16 x 128. $\endgroup$ Commented Dec 6, 2016 at 11:37
  • 1
    $\begingroup$ Additionally, I would add that he should check if he can make the network not to overfit, e.g. by having not 50 but only 1 neuron. Or 2. Just to see if that is really the problem or something else. $\endgroup$ Commented Dec 16, 2016 at 23:25

Have you tried early stopping? Hold out 20% of the data and stop training when the error on the hold-out set starts to increase. If you can't get high enough accuracy, increase the size of your model until you do.


One possible solution when you do not have enough data is to use Transfer learning.

This helps you to improve the performance of your model on the test data set. So, you can easily use one of the available pre-trained models in technical literature and update its weights based on your data. Take a look at this video. It is very helpful and you get a lot of ideas how to deal with your problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.