Keras/TF: Making sure image training data shape is accurate for Time Distributed CNN+LSTM

Question

The comprehensible data shape to me is like: (9186, 120, 120, 1) this means 9186 entry, of 120 by 120 pixel grey images. I learnt that using Time Distributed to design a CNN combined with an LSTM model, could learn more from images, knowing they are sequenced.

Following some tutorials, I found out I should expend another dimension which describes moving frames, in my case, I have 6 hours image, this would be good as a moving sequence, (probably I should try other lengths because time-series images are not of regular lengths, example:

8:00 img1 target1
9:00 img2 target2
10:00 img3 target3
11:00 img4 target4
12:00 img5 target5
13:00 img6 target6
19:00 img7 target7
   ...

My question is, when I expend another dimension of lets say length 6, how could such a model here output for every entrie exactly one prediction, but using a frame of length 6 to learn the next ?

train_images.reshape((1531, 6, 120, 120, 1)).shape

=> This gives me the impression, that such a model would output a prediction for a sequence of 6, which means 1531 result.

Am I understanding it wrong ?

[Edit following wind's answer]

I think my problem persists but I was not aware, I declared the first layer as

TimeDistributed( Conv2D(64, (3, 3), activation='relu'), input_shape=(6, 120, 120, 1) )

for x_train as

train_images_ = train_images.reshape((1531, 6, 120, 120, 1))

while y_train declared as

y_train_ = y_train.reshape((1531, 6, 8, 1))

as you can see x_train and y_train have the same length, with a frame of 6, image_length of 120, while, y_train consists of 8 target columns.

To my understanding, this is the way to follow with tensorflow, unfortuanatly, this is the error I am getting...

ValueError: Error when checking target: expected dense_2 to have 2 dimensions, but got array with shape (1531, 6, 8, 1)

Answer 1

As I understand your question correctly, you worry, that change of input shape (adding time-dimension) will affect your output shape. If so, you're wrong, the output is independent of input. The output shape depends only on the structure of a neural network.

Answer 2

I think you reshaped train_image in wrong way. for example if you have input like A1 A2 A3 A4 A5 A6 A7 A8... then input sequence should be like (to train 4 image at a time) A1 A2 A3 A4 -> set 1 A2 A3 A4 A5 -> set 2 A3 A4 A5 A6 -> set 3 and so on

where as your train image reshape is something like A1 A2 A3 A4 -> set 1 A5 A6 A7 A8 -> set 2