1
$\begingroup$

When we fit an LSTM model, each LSTM has a cell state which contains the information we want. I am wondering what's the dimension in the cell state (i.e. cell state itself should be a vector, then how large is the vector?)?

enter image description here (from http://colah.github.io/posts/2015-08-Understanding-LSTMs/ )

i.e. In the above architecture, how do we determine the dimension of the matrix Wc?

If I am using Keras package, how do I find the Cell state dimensions of each LSTM unit? Is it something that one can adjust?

Thank you!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

In Keras, the first argument in LSTM gives the dimensionality of the cell state.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Feel free to correct me but the first argument is "units", which is the number of LSTM neurons in the layer. Source: keras.io/layers/recurrent/#lstm $\endgroup$
    – pcko1
    Commented Jun 3, 2018 at 20:50
  • $\begingroup$ In this case, neurons = dimensionality. $\endgroup$
    – sssam
    Commented Jun 5, 2018 at 4:17
1
$\begingroup$

Notice that the next hidden state $h_t$ is produced by applying pointwise scalar operations to the next cell state $C_t$. (The precise formula $h_t = o_t * \tanh(C_t)$ can be found in the blog post you link to.) Therefore, the cell state and the hidden state have the same dimensionality.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.