I am working on a binary classification task using SVM. The dataset is quite large so I don't want to use k-fold CV for parameter tuning, but instead a simple train-validation-test split. I have done the following:
X_train, X_test, y_train, y_test = train_test_split( X, y, stratify = y, test_size=0.2, random_state=1) X_train, X_val, y_train, y_val = train_test_split( X_train, y_train, stratify = y_train, test_size=0.25, random_state=1)
So I have a 60-20-20 training-validation-test split. Since my validation set is predefined I want to use Sklearn's predefinedsplit. So I need to get the indices of my training and validation samples and set the validation indices to 0 and training indices to -1 so I tried the bottom answer of this question:
split_index = [0 if x in X_val.index else -1 for x in X_train.index]
But this returns a list of only -1's. I am unsure where I am going wrong. All my X and y _train, _val, _test are dataframes so .index can be applied. I have printed out X_train.index and X_val.index and both return Int64Index arrays of different lengths.
I also tried using Hypopt as one of the answers in the above link mentioned but it seems to be broken at the moment. Any suggestions as to how I should proceed?