1
$\begingroup$

I know concept of Epochs, batch size and iteration.

let's say,

Total_data = 6400

Batch_size = 64

Iteration = 100

In this, basically we are taking in 64 data points to computer memory and calculate and at each iteration we get weight updated. so after 100 iterations, we will fulfill one epoch.

My question is, after one epoch, we are using again the same 6400 data. How is it different from the first epoch in terms of learning? does the model select different 64 data point than first epoch in second epoch and try to learn? how does it internally really work?

I wish I can get some clear answers.

Thank you people in advance.

$\endgroup$

1 Answer 1

1
$\begingroup$

The data points are the same, but the model is different at each epoch. Weights are updated at each iteration so they will be different and in turn, the loss function will be different for the same data point at different epochs. What training process does is to change the weights slowly at each run to minimize the loss for the same data smaller and smaller.

Weights are modified in a way to minimize the error for the data points using an algorithm such as backpropagation in gradient descent. You might also want to check the explanation of backpropagation given by 3B1B in this youtube video.

$\endgroup$
1
  • $\begingroup$ Thank you. I will check the video! Best. $\endgroup$
    – S. Jay
    Commented Nov 16, 2019 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.