0
$\begingroup$

I am building an LSTM for price prediction using Keras. I am using Bayesian optimization to find the right hyperparameters. With every test I make, Bayesian optimization is always finding that the best batch_size is 2 from a possible range of [2, 4, 8, 32, 64], and always better results with no hidden layers. I have 5 features and ~1280 samples for the test I am trying.

Why is this the case? Is the lack of hidden layers due to the fact that I do not have many inputs and samples?

Also, literature always seems to suggest that a batch size of 2 is very low and a middle ground for the optimal batch_size is 32. How can I interpret this result?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

There are some papers which tell us that lower batch size may generalize better than large batch size. and large batch size may cause regularization in the model too. maybe that is the reason Bayesian optimization is suggesting a lower batch size for your dataset. Please check below papers,

https://openreview.net/pdf?id=B1eyO1BFPr

https://openreview.net/pdf?id=B1Yy1BxCZ

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.