1
$\begingroup$

Karpathy's' LSTM batch network LSTM batch network operates with batches

def checkSequentialMatchesBatch():
    """ check LSTM I/O forward/backward interactions """
    n,b,d = (5, 3, 4) # sequence length, batch size, hidden size
    input_size = 10
    WLSTM = LSTM.init(input_size, d) # input size, hidden size
    X = np.random.randn(n,b,input_size)
    #...

def checkBatchGradient():
    """ check that the batch gradient is correct """
    # lets gradient check this beast
    n,b,d = (5, 3, 4) # sequence length, batch size, hidden size
    input_size = 10
    WLSTM = LSTM.init(input_size, d) # input size, hidden size
    X = np.random.randn(n,b,input_size)
    #...

What does batch applied for? I'm only familiar with feeding one-hot word representation vector and can't understand LSTM learning process with batches. Please explain in terms of text processing.

Thanks in advance.

Improve this question
$\endgroup$
0
$\begingroup$

A batch is a grouping of instances from your dataset. For example a batch of 100 text samples that will be fed to train your model together.

Improve this answer
$\endgroup$
3
  • $\begingroup$ what do you mean under text samples? Can a batch, for example, contain a group of one-hot vectors and therefore be a one sentence? Or something more? $\endgroup$ – ichernob Mar 10 '17 at 18:39
  • $\begingroup$ The way you set up your feature space is up to you. For example, yes, you can use one-hot vectors as your feature space which will be the whole sentence. Then a batch would be multiple sentences. $\endgroup$ – JahKnows Mar 10 '17 at 19:21
  • $\begingroup$ I think this is better explained with images where it's more obvious what a single example should be. If you are considering all the pixels of a single image as its features. Then a batch would be taking multiple images simultaneously. $\endgroup$ – JahKnows Mar 10 '17 at 19:22
2
$\begingroup$

The accepted answer is correct, but it may also be helpful to think of a batch from a classification standpoint.

Suppose you have a binary classification problem that you are trying to solve using a multilayer perceptron, with 1000 examples of each class.

When training the model, you don't want to have to wait until the model has seen all of the data before a weight update is performed. That's computationally inefficient. Instead, you take, for example, 100 random examples of each class and call it a 'batch'. You train the model on that batch, perform a weight update, and move to the next batch, until you have seen all of the examples in the training set. One pass through the training set in this manner is called an 'epoch'.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.