Recursive Feature Elimination (RFE) with Logistic Regression and little correlation between the features and the target (SKLearn)

Question

I have little experience in the field of Machine Learning, so I apologise in advance for my ignorance and lack of intuitions.

I'm trying to build a classifying algorithm to identify if a subject is healthy or ill. I have the following data:

• 17 subjects.
• Each subject has 1140 features, ranging from -1 to 1.
• Each subject has a label from 0 to 6. 0 meaning very ill and 6 completely healthy.
• I only have data of subjects with labels from 0 to 2.
• The features (X) have low correlation with the labels (Y).

The way I obtained the features is irrelevant to this question, so I'm not going to go through it.

This is how the data looks:

Each colour represents a subject. As I have mentioned, I only have data of subjects with labels from 0 to 2, that is, Y=0, Y=1 or Y=2. And the features (X) are then 17 Subjects x 1140 features.

What I want to do is the following:

• Fit the data into a Logistic Regression.
• Use the Recursive Feature Elimination algorithm in order to fit the data into the classification function and know how many features I need to select so that its accuracy is high.
• Use Stratified Cross Validation to enhance the accuracy.

This is how I've implemented the algorithm in Python.

skf = StratifiedShuffleSplit(n_splits=50, test_size=0.3) # Cross-validation 50 times
estimator = LogisticRegression(C=1) # The estimator used is Logistic Regression
selector = RFECV(estimator, step=1, cv=skf, scoring="accuracy") # Run RFE
selector = selector.fit(X, Y) # Fit the data
print(selector.grid_scores_) # Print accuracy

And I obtained an accuracy around 60% for all the features, as shown below:

So, my question is. Do you think I implemented the code in the right way? Do you have any suggestion to enhance the accuracy? Maybe adjust a bit the parameters?

Thanks a lot. I hope I made myself clear.

Answer 1

Do you think I implemented the code in the right way?

Code is correct, but he is minimal possible.

The features (X) have low correlation with the labels (Y).

It's could be the biggest problem, features have to have correlation with labels.

Do you have any suggestion to enhance the accuracy?

• Make transformation of labels(X). Preparing of labels its one most important parts in Machine Learning. Please read data transformation or feature extraction.
• After this try different algorithms like xgboost, adaboost, random forest etc.
• Perhaps you should use regularization L1 or L2.
• Try early stopping.
• After this step use cross-validation.
• Choose suitable metrics for yours problem.