LSTM internals: How does an LSTM handle dimension reduction?

Question

I trained a Bidirectional LSTM of width 384 which took in an input of dimension (None, 4995, 12), using Tensorflow/Keras. I've been trying to understand how my input dimension is mapped to the $x_t$ in an LSTM cell input because I notice when I examine the internal weights of my LSTM that my weights are of dimension (384,12).

Does Tensorflow/Keras automatically divide up my input array into partitions of size 384? I assume that it doesn't time-slice my input because that is a feature of TimeDistributed LSTMs. Any information about this would be helpful.

UPDATE: I discovered when perusing a Tensorflow implementation of an LSTM that the input data is reshaped to be the same length as the $x_t$ array. My question, then, is whether the Tensorflow/Keras backend is implemented the same way.

def lstm(inputs, state, params):
    W_xi, W_hi, b_i, W_xf, W_hf, b_f, W_xo, W_ho, b_o, W_xc, W_hc, b_c, W_hq, b_q = params
    (H, C) = state
    outputs = []
    for X in inputs:
        X = tf.reshape(X, [-1, W_xi.shape[0]])
        I = tf.sigmoid(tf.matmul(X, W_xi) + tf.matmul(H, W_hi) + b_i)
        F = tf.sigmoid(tf.matmul(X, W_xf) + tf.matmul(H, W_hf) + b_f)
        O = tf.sigmoid(tf.matmul(X, W_xo) + tf.matmul(H, W_ho) + b_o)
        C_tilda = tf.tanh(tf.matmul(X, W_xc) + tf.matmul(H, W_hc) + b_c)
        C = F * C + I * C_tilda
        H = O * tf.tanh(C)
        Y = tf.matmul(H, W_hq) + b_q
        outputs.append(Y)
    return tf.concat(outputs, axis=0), (H, C)

Answer 1

The output of an LSTM is of the shape $[?,t,d]$ where $t$ is the number of timesteps and $d$ is the hidden dimension and there are $t$ number of cells in the LSTM - unless "return_sequences" is set to false in Keras. When "return_sequences" is false, the output is $[?,d]$ because it is returning the final timestep of what the output would have been if "return_sequences" had been set to true. This means that the input array is not being partitioned in any way - 384 was the hidden dimension.