What should be the value of batch_size in fit() method when using sgd (Stochastic Gradient Descent) as the optimizer?

Question

I am confused about the batch size of this model. I have used sgd i.e., Stochastic Gradient Descent as the optimizer (see the code). I am aware that in sgd, a single random instance from the training set is used to compute the gradient at each step. So, according to it, the batch_size should be equal to 1. Now, in the tf.keras.Sequential.fit() documentation it says:

If unspecified, batch_size will default to 32.

So, do I have to manually set the batch_size equal to 1? It is because the default value, 32 will make it a Mini-batch Gradient Descent.

    import tensorflow as tf
    from tensorflow import keras

    fashion_mnist = keras.datasets.fashion_mnist
    (X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()

    X_valid, X_train = X_train_full[:5000]/255.0, X_train_full[5000:]/255.0
    y_valid, y_train = y_train_full[:5000], y_train_full[5000:]

    model = keras.models.Sequential()

    model.add(keras.layers.InputLayer(input_shape = [28, 28]))
    model.add(keras.layers.Flatten())
    model.add(keras.layers.Dense(300, activation = "relu"))
    model.add(keras.layers.Dense(100, activation = "relu"))
    model.add(keras.layers.Dense(10, activation = "softmax"))

    model.compile(loss = "sparse_categorical_crossentropy", optimizer = "sgd", metrics = ["accuracy"])

    history = model.fit(X_train, y_train, epochs = 30, validation_data = (X_valid, y_valid))

Answer 1

Batch size specify the number of observations used to adjust the parameters for each iteration. If it is 1, the result from this observation will be used. If it is more than 1, average performance will be used.

Ideally you should consider batch size as a hyperparameter. Which means that you should determine the optimal batch size for your problem. You may use a simple for loop or grid/random search with other hyperparameters.