So every time I try to write a CNN or RNN I have problems understanding how the dimensions work.
I don't understand why does the model compile, which in my opinion means that the output dimension of the previous layer is the input dimension of the next one and so on(the layers are compatible with each other). So how is it possible that when I fit my model I get an error in the 3rd or 4th layer(or any other layer except the first one) saying that for ex it expects 3 dimensions but got 2.
On a more precise example, lets say I have the following one-shot model:
vocab_size =10000 src_txt_length =10 sum_txt_length = 100 inputs = Input(shape=(src_txt_length,)) encoder1 = Embedding(vocab_size, 128)(inputs) encoder2 = LSTM(128)(encoder1) encoder3 = RepeatVector(sum_txt_length)(encoder2) decoder1 = LSTM(128, return_sequences=True)(encoder3) outputs = TimeDistributed(Dense(vocab_size, activation='softmax'))(decoder1) model = Model(inputs=inputs, outputs=outputs) model.compile(loss='categorical_crossentropy', optimizer='adam') model.fit(X,y)
I get an error in the TimeDistributed layer saying that it expects 3 dimensions but got 2.
X.shape == (20000,10) len(y) == 20000 # y is a list of labels 0s and 1s
So if I want to calculate the dimension of each layer when I fit my data on paper, how will I do that ?