What is the difference between the test and training data sets? As per blogs and papers I studied, what I understood is that we will have 100% data set that is divided into 2 sets (test data set is 30% and reaming 70% is training data set). I would like to know more points and use of differentiating the 100% data set to test and training data sets.
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
6
In Machine Learning, we basically try to create a model to predict on the test data. So, we use the training data to fit the model and testing data to test it. The models generated are to predict the results unknown which is named as the test set. As you pointed out, the dataset is divided into train and test set in order to check accuracies, precisions by training and testing it on it.
The proportion to be divided is completely up to you and the task you face. It is not essential that 70% of the data has to be for training and rest for testing. It completely depends on the dataset being used and the task to be accomplished. For example, a simple dataset like reuters. So, assume that we trained it on 50% data and tested it on rest 50%, the precision will be different from training it on 90% or so. This is mostly because in Machine Learning, the bigger the dataset to train is better. You can refer this paper, which tells the precision values based on the dataset size. It now depends on you, what precision or accuracy you need to achieve based on your task.
This said so, how would you predict the results for which you do not have the answer? (The model is ultimately being trained to predict results for which we do not have the answer). I would like to add on about validation dataset here.
Sets:
Training Set: Here, you have the complete training dataset. You can extract features and train to fit a model and so on.
Validation Set: This is crucial to choose the right parameters for your estimator. We can divide the training set into train set and validation set. Based on the validation test results, the model can be trained(for instance, changing parameters, classifiers). This will help us get the most optimized model.
Testing Set: Here, once the model is obtained, you can predict using the model obtained on the training set.
Refer this for more info.
-
$\begingroup$ (+1) for also explaining about the validation set :) $\endgroup$– Dawny33Jul 6, 2016 at 7:26
-
$\begingroup$ @Hima Varsha Thank you ,now I have clear view :) some of my questions are not yet answered.If possible plz find some time to answer them :) Good day $\endgroup$ Jul 6, 2016 at 7:59
-
$\begingroup$ So the use of dividing the dataset into training and test is to predict using our model and test using the test set. This will help us to check precisions of our model. Test dataset should not be used to fit the model to prevent bias $\endgroup$ Jul 6, 2016 at 8:04
-
$\begingroup$ @DilipBobby Is there anything more specific I can help you with? $\endgroup$ Jul 6, 2016 at 8:12
-
$\begingroup$ @Hima Varsha plz share your answer (datascience.stackexchange.com/questions/12584/…) 2) datascience.stackexchange.com/questions/12622/… $\endgroup$ Jul 6, 2016 at 8:20
$\begingroup$
$\endgroup$
^ I completely agree with the answer above from Hima Varsha, however I wanted to add that sometimes there is different names for testing sets. A data science company I used to work for would use a training set, validation set, testing set, and sometimes a third testing set called a "holdout set". I'm not sure if that was particular to the company I was working for, but you might see it crop up in literature or documentation in the future. Also, there is a third metric that almost always goes along with precision and recall called the F1 score. This is actually calculated from both precision and recall, but commonly used as a summary of both precision and recall.
