0
$\begingroup$

I have a question about how to correctly setup a Keras Tuner model for a stacked LSTM model. What I have tried is the normal tutorial with a loop and the hp.Int() function to define the size of each LSTM layer like this:

for i in range(hp.Int('num_lstm_layers', min_value=1, max_value=4)):
   lstm_units_i = hp.Int('units_' + str(i), min_value=2**6, max_value=2**9, step=2**5) 
   lstm = Bidirectional(LSTM(lstm_units_i, return_sequences=True), name=f'lstm_{i}')(lstm)

However, with this configuration the second and third layer could have more units compared to the first layer (and so on) - this is not something we would want, is it? We would want the layers to get smaller and force the model to learn on less data? The reason i ask is, because the 'best' model afterwards was one, where the second layer was double of the first layer for example, and I dont know if this is 'correct' for LSMTs?

I also have created a for loop, whit the condition that the next layer can only be as deep as the layer beforehand like this:

for i in range(hp.Int('num_lstm_layers', min_value=1, max_value=4)):

        if lstm_units_prev is not None:
            lstm_units_i = hp.Int('units_' + str(i), min_value=2**6, max_value=lstm_units_prev, step=2**5)
        else:
            lstm_units_i = hp.Int('units_' + str(i), min_value=2**6, max_value=2**9, step=2**5)
            
        lstm = Bidirectional(LSTM(lstm_units_i, return_sequences=True))(lstm)
        lstm_units_prev = lstm_units_i

Would this be the preferred way, or does it not matter?

Thanks,

Barry

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.