When should I do train test split and feature selection if my dataset is unbalanced?

Question

I have a dataset containing ECG signals with 5 different classes describing the quality of a particular window of the ECG signal. I need to build a machine learning model to predict the signal quality based on features extracted from each window.

The dataset contains 1020, 5-second windows, with the following label distribution:

• Very Good: 485 occurrences
• Good: 272 occurrences
• Moderate: 138 occurrences
• FL: 75 occurrences
• Bad: 50 occurrences

The dataset is imbalanced, so I haven't performed feature selection yet. I learned that feature selection should be done before data augmentation to ensure that the synthetic data created to balance the dataset will influence the feature significance. However, I also read that the train and test split should be done before feature selection. I'm concerned that if I split the data, the lack of minority data will affect the feature selection process.

I'm new to machine learning, so I'd appreciate any suggestions on the right way to approach this. Any further suggestions would be really helpful.

Answer 1

Train/test split should be done before feature selection so that the selection of features for the model is independent of the test set (therefore, you can test the quality of the feature selection as well).

If you are concerned about sufficient prevalence of individual label classes, you can perform stratified sampling for the train/test split to ensure you have enough of each class in both sets.

Answer 2

Given Problem :- 

This problem is about building a machine learning model to predict ECG signal quality from imbalanced data, where feature selection, data augmentation, and train-test splitting need to be carefully handled to avoid bias in the minority classes during feature selection and ensure a valid model

Solution:-

From the given above problem I would like to suggest a multi class classification algorithm in machine learning with neural networks through Frameworks such as TensorFlow / PyTorch making this a better approach to achieve building your machine learning algorithm.

The life cycle of the machine learning model building involves :-

1.Data collection :- the first step which u have already achieved.

2.Data Preprocessing :- the data collected is looked for null values and required following steps such as :-

1.  checking for null values 
2.  Removing the null values
3.  Data modelling 
4.  Data Labelling
5.  Running exploratory data analysis

3.Feature selection:- from exploratory data analysis select the required features and Standardise the features to have better model results.

4.Model Building :- Take the train test split based upon the required need which can be represented below

Train	Test
70	30
80	20
60	40

after choosing the right train test method build the model through `PyTorch/ tensor flow/ SKLearn frameworks`.

5.Model Evaluation :- once the building of the model is done evaluate on the built model

6.Model optimisation :- once the building of multiclass classification of the ECG scans is done move forward with checking for accuracy based on accuracy take the decision to do following steps:

1. Use ensemble methods to increase the accuracy and fitting of the model 
2. Use adversial training or other different model optimisation techniques  to make the work better

7.Model deployment:- if this is a end to end application then make sure that u create the front end design and then implement the model analysis in the backend and deploy it using cloud providers and docker.