I want to train my neural net to overfit the training data. Should I just keep fitting my model with the same training data by using K-fold validation and setting epochs to infinity? Then, after it reaches nearly 100 percent prediction, I can just stop the training. Is it the right way to do it?
If you want to overfit, then yes you just need to keep fitting the training data through your network until you reach as close to zero training loss as possible (note that zero loss is stronger than 100% prediction, and will result in a greater amount of overfitting). There is no need to use cross validation. If your network is sufficiently complex, then it won't take very long to reach very high accuracy on the training data.
It is very important to understand the difference between overfitting / underfitting and bias error / variance error.
Three things to have high probability to overfitting:
- Complex model with very large number of parameters , there is a relation between the number of samples used for training and the number of parameters. Read about optimism
- Few number of training samples
- Training for large number of iterations, until you have training error very close to zero
It is very important to know that these things increase the probability to have overfitting, but another important factor is the problem itself. If you have two classes that are separable using your feature space, even with all these conditions you may not overfit, even if the training error is zero.