# LSTM internals: How does an LSTM handle dimension reduction?

I trained a Bidirectional LSTM of width 384 which took in an input of dimension (None, 4995, 12), using Tensorflow/Keras. I've been trying to understand how my input dimension is mapped to the $$x_t$$ in an LSTM cell input because I notice when I examine the internal weights of my LSTM that my weights are of dimension (384,12).

Does Tensorflow/Keras automatically divide up my input array into partitions of size 384? I assume that it doesn't time-slice my input because that is a feature of TimeDistributed LSTMs. Any information about this would be helpful.

UPDATE: I discovered when perusing a Tensorflow implementation of an LSTM that the input data is reshaped to be the same length as the $$x_t$$ array. My question, then, is whether the Tensorflow/Keras backend is implemented the same way.

def lstm(inputs, state, params):
W_xi, W_hi, b_i, W_xf, W_hf, b_f, W_xo, W_ho, b_o, W_xc, W_hc, b_c, W_hq, b_q = params
(H, C) = state
outputs = []
for X in inputs:
X = tf.reshape(X, [-1, W_xi.shape])
I = tf.sigmoid(tf.matmul(X, W_xi) + tf.matmul(H, W_hi) + b_i)
F = tf.sigmoid(tf.matmul(X, W_xf) + tf.matmul(H, W_hf) + b_f)
O = tf.sigmoid(tf.matmul(X, W_xo) + tf.matmul(H, W_ho) + b_o)
C_tilda = tf.tanh(tf.matmul(X, W_xc) + tf.matmul(H, W_hc) + b_c)
C = F * C + I * C_tilda
H = O * tf.tanh(C)
Y = tf.matmul(H, W_hq) + b_q
outputs.append(Y)
return tf.concat(outputs, axis=0), (H, C)

• Are you still looking for an answer to this? Jun 22, 2022 at 9:52
• @hH1sG0n3 I think I understand LSTMs much better now, as indicated below. No additional answer needed unless you have a clarification Jul 5, 2022 at 20:11

The output of an LSTM is of the shape $$[?,t,d]$$ where $$t$$ is the number of timesteps and $$d$$ is the hidden dimension and there are $$t$$ number of cells in the LSTM - unless "return_sequences" is set to false in Keras. When "return_sequences" is false, the output is $$[?,d]$$ because it is returning the final timestep of what the output would have been if "return_sequences" had been set to true. This means that the input array is not being partitioned in any way - 384 was the hidden dimension.