Usually in a LSTM network, we have certain parameters that need to be set before the model can begin training. I am specifically talking about vocabulary size, padding length and embedding dimension. Below is a simple LSTM network where I have randomly chosen the 3 parameters:
vocab_size = 20000 pad_size = 35 embedding_dim = 50 # ONE HOT REPRESENTATION FOR TRAINING SET ohr_train = [one_hot(i, vocab_size) for i in train_x] # PAD EACH TEXT IN TRAIN SET SO THAT EACH TEXT IS OF SAME LENGTH train_embedded_docs = pad_sequences(ohr_train, padding = 'pre', maxlen = pad_size) # DEFINE THE MODEL model = Sequential() model.add(Embedding(vocab_size, dimension, input_length = pad_size)) model.add(Dense(1, activation = 'sigmoid'))
My question is how do you set all the three above mentioned parameters?
PS: From the answers I understood how to set vocabulary size. Padding length should be more than the maximum length of text in train set (not the whole set as that would lead to data leakage!).
But when setting the embedding dimension via HP tuning, it will be a time consuming process as for each combination, I would have to run the whole model and as you know neural nets take a long time to run. Isn't there a better way?