1
$\begingroup$

I am trying to figure out if it makes sense that the width of the network could be smaller than the input/output size?

So for example, I am giving the Neural Network 2048 numbers, and I am expecting 2048 numbers back. I would also like to use LSTM's, which take a lot of time/space etc to train, and having one or a few 2048 or larger LSTM layers connected to Dense layers would take a lot of space/time to train. But maybe that is the way to go?

Does anyone have any experience with this type of problem? Thank you!

$\endgroup$
  • $\begingroup$ Would you please elaborate? Do you want to use an LSTM or a simple dense layer? Is your data temporal making you try LSTM? $\endgroup$ – Media May 7 '19 at 13:23
  • $\begingroup$ Yes, the batches consist of sequences. The LSTM network is not 'stateful', and with the LSTM cells I am expecting it to learn the relationships between the sequences and predict them. :) $\endgroup$ – Space Ghost May 7 '19 at 13:29
1
$\begingroup$

In high dimensional spaces like the one-dimensional space, you have to employ a very simple network, maybe just a single neuron and investigate whether your data is linearly separable in that feature space which can not be visualized. If you observe that you do not have a good performance, you can increase the size or the layers of your network step by step. You can generalise what I've just referred to other networks like RNNs.

What I've referred means that your data may be in a way that is separable though it's in 1000 dimensional space and you just need one neuron to classify it.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.