I am using gene expression data that are float numbers and want to train classifiers in view of binary classification. Since I am a novice in this field I have some questions:
- The first classifier I am using is
SVM. I am using
sklearntools which require a split of the data set in training and test data sets. As far as I know, in order to build the model one needs a splitting of the data set in train and validation data sets (finding the parameters of the model), and for fine-tuning of hyperparameters, one will need a test data set. Interestingly, given what I found in going through
sklearndocumentation, there is suggested a split in train and test data set only. There is no assertion on the validation data set. So, I am doubtful, If I am running the classifier correctly.
Here is the code that I am using:
from sklearn.model_selection import train_test_split from sklearn.utils import shuffle from sklearn import svm from sklearn.metrics import roc_curve, auc xall, yall = shuffle(x_sm, y_sm, random_state=21) x_train, x_test, y_train, y_test = train_test_split(xall, yall, test_size=0.3, random_state=3, stratify=y_sm) x_train.shape, x_test.shape` classifier = svm.SVC(kernel='linear', gamma='auto', C=2,probability=True) classifier.fit(x_train, y_train) y_predict = classifier.predict(x_test) probs= classifier.predict_proba(x_test) fpr, tpr, threshold = roc_curve(y_test, preds)
Can somebody explain, if implicitly
sklearn is using internally the triple split in training, validation and test data sets ? If not, how should I modify the code to include the known scheme of splitting in train, validation and test data sets ?
If instead of random splitting, one uses k-fold cross-validation, will the result again be a splitting in two and not three data sets ?
Before training the model, I am using the standardization tools and PCA for feature and thus dimensionality reduction. After that, I am taking the first 10 PCA-components in training the model as described above. Is this the correct way one would suggest ? Apart from PCA, there are other dimensionality reduction tools. Should one use a few of them, train the model and decide at the end, based on the model performance, which of the dimensionality reduction tools to use for a particular classifier?
SVM, I would like to use 3 more classifiers on the same data set and compare their performance. Given the nature of the data I have, which classifier should I choose?
I will highly appreciate your answers. Thanks.