What are the steps and correct order of the operations in Machine Learning? [from Getting data to optimising models]

Question

I've followed lots of tutorials on Machine Learning but in each of these, they go for a different strategy so it's quite confusing for me. I want to Know that what are the operations involved and what are the correct ordering of these.

AS of now, I think the process and the ordering are ->

1. Get Data
2. Delete Duplicates
3. Find Missing Values and Outliers
4. Create New Features
5. Deal with missing values and Outliers
6. Build a base model
7. Find the best features to select
8. Try and find different Models
9. Select the BEST model
10. Hypertuning of the Model

Please Do Provide if something is missing and correct the sequence.

Answer 1

Yes learning online from different tutors can lead to different strategies. But this stats that the flow depends upon the type of data. Not everytime you have to create features or deal with outliers.

In case of images you reduce noise by applying different methods & filters, in case of text you remove stopwords, punctuation etc. So it depends upon data & its amount. General way to proceede for me is as follow

1. Understand definition & get domain knowledge
2. Gather data
3. Pre-process it (Involves removing noise, applying transformation, pruning data etc)
4. Do feature selection if necessary
5. Do data visualisation
6. Do different model experiments with sample data.
7. Fix 2 3 models for case
8. Train with real data
9. Use different matrices to evaluate (depends upon the data & definition)
10. Make model accessible using API/Direct to client

This is how in industry i have seen everyone around me works & i follow the same. Has helped me but this is way general, here you have much to change based upon definitions & tasks.

Answer 2

Below steps are similar to the other suggestions:-

1. Understand the problem statement, relate it with the data.
2. Read the data from the file (CSV or any format is given) and find out the number of columns and rows in it.
3. Identify features with null values because we need to handle them first. Once identified the null values in the data, find out measures how you will fix the null values, and fix the null values.
4. Do the same operation (step no 3) for the test data as well (If you have 2 separate train and test datasets).
5. Now, wherever you have columns with datatype other than integer or float, you must convert that data into type numeric.
6. After that, you need to select the features that you will be using for the training of the model.
7. Here you may also create new features depending upon your approach and requirement.
8. After that, we can start training the model. So we select few Models for it (say 2-3)
9. After training is done, we predict the values of the test data and see how accurate our model is performing (Accuracy for Classification problems).
10. Now, this accuracy will decide what change is needed with the features and Model selection.
11. In the case of features, you can check the importance of the features and take the decision of dropping/including new features.
12. Also, to improve the performance/accuracy we can do hyperparameter tuning of the parameters and repeat the step 8-10