I am building a "predict next word" model using the following model architecture. The codes fine, but I have a few questions:
# define model model = Sequential() model.add(Embedding(vocab_size, 50, input_length=seq_length)) model.add(LSTM(100, return_sequences=True)) model.add(LSTM(100)) model.add(Dense(100, activation='relu')) model.add(Dense(vocab_size, activation='softmax')) print(model.summary())
Between the Embedding and LSTM layers, are there 50 connection weights, or 50 * 100 connection weights? Based on the documentation, this LSTM block contains 100 cells. So I am not sure how exactly does the Embedding layer connect to the LSTM layer?
Also, between LSTM(100) layer and the Dense(100, activation='relu') layer, are there 100 connection weights or 100 * 100 connection weights?