Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have been playing around the NLTK algorithm for some data prediction.
Starting from this gib, I started my understanding process. However, there are some bits that don't make sense.
If I have a set of 100 features, all classified, what's the sense to split them, take 10% and build the training set on that alone? I thought the training set should have included all the list, and accuracy is measured against the new keywords being tested against?
Any hint would be helpful.
If you don’t have a classified testing set - which allows to measure a performance score, then it is useful to use part of your training data as a validation set, meaning that you test the performance of your model on it (because you have the true labels/values).
The percentage of the split is arbitrary and depends on the degrees of freedom you have (number of individuals vs number of variables) - 0.1 to 0.25 is a common choice.
