What is a batch in machine learning?

Question

Karpathy's' LSTM batch network LSTM batch network operates with batches

def checkSequentialMatchesBatch():
    """ check LSTM I/O forward/backward interactions """
    n,b,d = (5, 3, 4) # sequence length, batch size, hidden size
    input_size = 10
    WLSTM = LSTM.init(input_size, d) # input size, hidden size
    X = np.random.randn(n,b,input_size)
    #...

def checkBatchGradient():
    """ check that the batch gradient is correct """
    # lets gradient check this beast
    n,b,d = (5, 3, 4) # sequence length, batch size, hidden size
    input_size = 10
    WLSTM = LSTM.init(input_size, d) # input size, hidden size
    X = np.random.randn(n,b,input_size)
    #...

What does batch applied for? I'm only familiar with feeding one-hot word representation vector and can't understand LSTM learning process with batches. Please explain in terms of text processing.

Thanks in advance.

Answer 1

A batch is a grouping of instances from your dataset. For example a batch of 100 text samples that will be fed to train your model together.

Answer 2

The accepted answer is correct, but it may also be helpful to think of a batch from a classification standpoint.

Suppose you have a binary classification problem that you are trying to solve using a multilayer perceptron, with 1000 examples of each class.

When training the model, you don't want to have to wait until the model has seen all of the data before a weight update is performed. That's computationally inefficient. Instead, you take, for example, 100 random examples of each class and call it a 'batch'. You train the model on that batch, perform a weight update, and move to the next batch, until you have seen all of the examples in the training set. One pass through the training set in this manner is called an 'epoch'.