I am working on a project where I analyse MEG data. I have 102 channels as a vector and a 2D matrix of the channels (11x14) to show spatial relations - I want to include that in the AI architecture. But I also want to have some time context for the spatial data.

So far my idea is to have a sort of "sliding window" over the data where I consider some time steps. Then what I can do with it is:

  1. Have some parallel CNNs, one for each time step, whose output gets combined to a single feature vector. That vector is then concatenated with the hidden state from the LSTM part. That part is an LSTM for each time step that feeds into the next step. This is essentially the idea of this paper (figure 3) - Emotion Recognition from Multi-Channel EEG through Parallel Convolutional Recurrent Neural Network

CNN + LSTM architecture (Here i consider 2 time steps at a time)

  1. A similar idea as above, but instead of having an LSTM for each time step, I model the data so as to have only one input of the shape (samples, time steps, data). I kind of understand what happens inside a single LSTM cell but I don't know what would be better on the architecture level.

  2. A new solution, where instead of using normal CNNs I use the new ConvLSTM2D layer in tensorflow. This does the job of an LSTM but on a 2D input. Here I would only have a single input of the shape (samples, time steps, rows, cols, channels). I am not sure, however if it would learn the spatial relations as well?

ConvLSTM2D based architecture (here i consider 10 time steps at a time)

Are any of these solutions completely stupid? Does any of them make much more sense than the others? Also any link to like a tutorial on LSTM-based architecture principles or smth like that would be much appreciated!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.