0
$\begingroup$

I have been training a convolutional neural network on emotion detection. Now, I would like to extract features for my data to train an LSTM layer. In my case, the top convolutional layers in the network has the following dimensions: [None, 4, 4, 512] and [None, 4, 4, 1024]. Therefore, this will give a total of 8192 and 16384 dimensional vectors. This is too large to train an LSTM layer. Therefore, I would like to know what is the best possible way to reduce the dimensionality of this vector? In other words, should I apply global average pooling to the conv layer after obtaining the activation or any other dimensionality reduction technique? In this case, my features will be a vector of 512 or 1024 dimensions, which makes sense.

Any help is much appreciated!!

$\endgroup$
1
$\begingroup$

Applying a pooling layer following a convolution is a standard way to reduce the size of the input matrix and get the invariant features. You might also want to consider adding a dense layer with a smaller number of output neurons.

| improve this answer | |
$\endgroup$
  • $\begingroup$ So should I apply max pool, avg pool or what exactly? Thank you @Anshul for your answer $\endgroup$ – I. A Nov 2 '18 at 15:11
  • $\begingroup$ Typically the type of pooling to be used is application specific. You might want to try both and see what works better. $\endgroup$ – Anshul G. Nov 3 '18 at 11:50
0
$\begingroup$

1- Applying multiple convolution layers result in more sophisticated/rich features extraction. This will also reduce the dimensions of the feature maps.

2- Max-Pooling after convolutions is also an excellent way of extracting the most impacting features from an overall set of feature maps.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.