I have been doing DS for a couple of years now and have returned to "tinkering" a bit more with toy data sets and overall just honing my skills a bit.
I was recently playing with a very straightforward dataset that contains the history of 100 meter foot races (like in the Olympics). The data is not complex, just things like athlete height, age, etc.
I input the data into a dense network (Keras) and I was getting poor-to-fair results. However, this was all with a batch size of 1 but everything changed as soon as I worked with other batch sizes (2,4,8,16, etc). All my metrics went from poor to outstanding, even with just a batch size of 2.
Why is this? What is the layman's explanation for the effect batch sizes can have on a dense NN?