0
$\begingroup$

Lets assume I have 103 training examples. I want a mini-batch to be of the size 16. That means that there will be 6 mini-batches of the size 16 and one mini-batch of the size 7.

In the tensor flow one needs to specify the shape of the input:

x = tf.placeholder(tf.int32, shape=[batch_size], name='x')

which of course led to the following error:

Cannot feed value of shape (7,) for Tensor 'x_10:0', which has shape '(16,)'

So, what do I do with the mini-batch of size 7? Should I find the corresponding mini-batch size, that will create equal mini-batches? If so, how can I follow then the advice to create mini-batches of the power of two? or I should disregard the last mini batch?

$\endgroup$
0
$\begingroup$

A few basic approaches come to mind. It depends on your context and intention which might work best for you. Here are some ideas assuming that working with a partial batch is not an option:

  1. Drop the remainder. This might make more sense when you have many complete batches, and you consider the set of examples to be enough.

  2. Complement the final batch with duplicated samples from the other batches. E.g. randomly get one example from each batch.

  3. If something like that is feasible at all, generate additional (possibly synthetic) examples to complete the final batch.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.