I'm new to data science. I'm trying to get the best model for Random Forest. Unfortunately, I'm not sure if my idea can produce a good generalized model.
1) I have split data to TrainingSet (70%) and TestSet (30%)
2) Then randomly selected hyperparameters for RandomForest and a number of folds for CrossValidation between (2-15)
3) Then I fetch the TraingSet data to the RandomForest learner
4) Then do CrossValidation of the model - from CrossValidation I'm getting array with predictions
5) Measure Accuracy of prediction from CrossValidation against targets from the TrainSet
6) Repeat all steps and try minimize the AccuracyError
Is this a good way to get best generalized model?
Do I need to split data into TrainSet and TestSet?
OR I should I search for the optimal hyperparameters and number of folds with all data? I have feeling I don't need to split data when using k-fold CrossValidation during Hyperparameters tunning.