Does it make sense to train a CNN as an autoencoder?

Question

I work with analyzing EEG data, which will eventually need to be classified. However, obtaining labels for the recordings is somewhat expensive, which has led me to consider unsupervised approaches, to better utilize our quite large amounts of unlabeled data.

This naturally leads to considering stacked autoencoders, which may be a good idea. However, it would also make sense to use convolutional neural networks, since some sort of filtering is generally a very useful approach to EEG, and it is likely that the epochs considered should be analyzed locally, and not as a whole.

Is there a good way to combine the two approaches? It seems that when people use CNN's they generally use supervised training, or what? The two main benefits of exploring neural networks for my problem seem to be the unsupervised aspect, and the fine-tuning (it would be interesting to create a network on population data, and then fine tune for an individual, for instance).

So, does anyone know if I could just pretrain a CNN as if it was a "crippled" autoencoder, or would that be pointless?

Should I be considering some other architecture, like a deep belief network, for instance?

Answer 1

Yes, it makes sense to use CNNs with autoencoders or other unsupervised methods. Indeed, different ways of combining CNNs with unsupervised training have been tried for EEG data, including using (convolutional and/or stacked) autoencoders.

Examples:

Deep Feature Learning for EEG Recordings uses convolutional autoencoders with custom constraints to improve generalization across subjects and trials.

EEG-based prediction of driver's cognitive performance by deep convolutional neural network uses convolutional deep belief networks on single electrodes and combines them with fully connected layers.

A novel deep learning approach for classification of EEG motor imagery signals uses fully connected stacked autoencoders on the output of a supervisedly trained (fairly shallow) CNN.

But also purely supervised CNNs have had success on EEG data, see for example:

EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces

Deep learning with convolutional neural networks for brain mapping and decoding of movement-related information from the human EEG (disclosure: I am the first author of this work, more related work see p. 44)

Note that the EEGNet paper shows that also with a smaller number of trials, purely supervised training of their CNN can outperform their baselines (see Figure 3). Also in our experience on a dataset with only 288 training trials, purely supervised CNNs work fine, slightly outperforming a traditional filter bank common spatial patterns baseline.

Answer 2

Yes, you can use a convolutional network in an autoencoder setup. There is nothing strange with it. People have problems figuring out deconvolution layers, though.

Here you can find an example of a convolutional autoencoder using Keras framework: https://blog.keras.io/building-autoencoders-in-keras.html