I am bit confused about validating data. What is this data mainly for?? Like I am seeing some tutorial and they have some training[I know it] images , they they have some validation images[donot know] and some testing[I know] images? So what is validating images mainly for??
1
-
$\begingroup$ Not sure if this is what you are looking for stats.stackexchange.com/questions/19048/… $\endgroup$– Vikas MadhusudanaCommented May 24, 2016 at 11:05
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
There are two uses for the validation set:
1) Knowing when to stop training
Some models are trained iteratively - like neural nets. Sooner or later the model might start to overfit the training data. That's why you repeatedly measure the model's score on the validation set (like after each epoch) and you stop training once the score on the validation set starts degrading again.
From Wikipedia on Overfitting:
"Training error is shown in blue, validation error in red, both as a function of the number of training cycles. If the validation error increases (positive slope) while the training error steadily decreases (negative slope) then a situation of overfitting may have occurred. The best predictive and fitted model would be where the validation error has its global minimum."
2) Parameter selection
Your model needs some hyper-parameters to be set, like the learning rate, what optimizer to use, number and type of layers / neurons, activation functions, or even different algorithms like neural net vs SVM... you'll have to fiddle with these parameters, trying to find the ones that work best.
To do that you train a model with each set of parameters and then evaluate each model using the validation set. Finally you select the model / the set of parameters that yielded the best score on the validation set.
In both of the above cases the model might have fit the data in the validation set, resulting in a biased (slightly too optimistic) score - which is why you evaluate the final model on the test-set before publishing its score.
