I am currently reading Deep Learning with Python by Francois Chollet, the author of Keras, and in one of his definitions for Mini-batch, he explains that the power of 2 for the batch_size
is due to memory allocations in gpu/ Could anyone elaborate on this?
Mini-batch or batch—A small set of samples (typically between 8 and 128) that are processed simultaneously by the model. The number of samples is often a power of 2, to facilitate memory allocation on GPU. When training, a mini-batch is used to compute a single gradient-descent update applied to the weights of the model.