1
$\begingroup$

i have a question about the tutorial of tensorflow to train the mnist database how do i create my own batch without using next_batch() , the idea is to train with a batch of 50 ,then 100 and so but it has to be in order

here is the code

cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
sess.run(tf.initialize_all_variables())
for i in range(20000):
    batch = mnist.train.next_batch(50)#revisar aqui
    if i%100 == 0:
        train_accuracy = accuracy.eval(feed_dict={x:batch[0], y_: batch[1], keep_prob: 1.0})
        print("step %d, training accuracy %g"%(i, train_accuracy))
    train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})
$\endgroup$
1
$\begingroup$

In a nutshell

If you have a look at what mnist.train is you'll find there are two numpy arrays in it: mnist.train.images (shape (55000, 784))and mnist.train.labels (shape (55000, 10)).Then all you have to do is iterate on these.

Example

This example is voluntarily a bit complicated so that it is robust to whatever sizes you chose but keeps the idea of fixing a number of batches as primary information on how much you want to train on the dataset :

X_train = mnist.train.images
Y_train = mnist.train.labels
number_of_examples = X_train.shape[0]
batch_size = 1000
number_of_batches = 200
epoch = 0

for i in range(number_of_batches):
    

    j = (i - epoch) * batch_size % number_of_examples
    k = (i - epoch + 1) * batch_size % number_of_examples

    if (k < j):
        k = number_of_examples
        batch_x = X_train[j:number_of_examples, :]
        batch_y = Y_train[j:number_of_examples, :]

        print('Shuffling data to retrain on dataset')
        data = numpy.concatenate((X_train, Y_train), axis=1)
        np.random.shuffle(data)

        # redefine X_train and Y_train from shuffled dataset
        X_train = data[:, :784]
        Y_train = data[:, 784:794]

        epoch = i + 1
    else:
        batch_x = X_train[j:k, :]
        batch_y = Y_train[j:k, :]

    # [...]
    train_step.run(feed_dict={x: batch_x, y_: batch_y, keep_prob: 0.5})

j is the row of the dataset which will be the batch's first row k is the last one, so j-k=batch_size examples per batch, as expected. Except if you've reached the end of the dataset (k<j) in which case you just go to its end and then shuffle the dataset so that batches won't be the same.

You count i - epoch and not just i so that you're sure you'll always start at the 1st row of the dataset even when you've already gone completely through it. For instance if number_of_examples = 55555, batch_size = 1000 and number_of_batches = 200: i = 56 is the first time you go again through the (shuffled) dataset but 56 * 1000 % 55555 = 445 so you'd start at row 445 and not 0.

Also you HAVE to concatenate, otherwise the pairs (example, label) will be lost.


i, j ,k % Whaaat?

Run the toy code below to see how the iteration index behaves:

number_of_examples = 55555
batch_size = 1000
number_of_batches = 200
epoch = 0

for i in range(number_of_batches):

    j = (i - epoch) * batch_size % number_of_examples
    k = (i - epoch + 1) * batch_size % number_of_examples

    if (k < j):
        print('Overlap : ', j, number_of_examples)
        epoch = i + 1
    else:
        print(i, '   ', j, k)
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.