General Machine Learning Workflow Question

Question

I'm very new to machine learning and want to understand the general process by which it is carried out. I've worked through the famous 'iris' tutorial and want to ask if the principles in that tutorial are applicable to every future machine learning project I undertake.

I work in biological sciences, and am interested in applying machine learning algorithms to biological sequence data as away of categorizing classes or doing unsupervised clustering. From my understanding, each project starts with:

1. Me defining the 'scope' or aim of what I want to learn from the raw data.
2. Generating features/attributes from my OWN code/algorithms that could potentially differentiate between the two classes
3. This basically generates a huge matrix of x_features by y_entries
4. Feed the matrix into a machine learning algorithm (I'm sure this is vastly oversimplified).

To give an example, say I have 10,000 protein sequences, and I believe 5000 are 'Class1' and 5000 are 'Class2', but I do not know how to differentiate them by eye. I need to generate x_features (in some informed way) of this sequence using my own custom algorithms, and feed the resulting 10000 entries into an algorithm.

Is this the right approach? I'd be eternally grateful if someone could direct me towards a beginners tutorial that revolves around analyzing biological sequence data.

Answer 1

I agree with most of the answer. However, I think you are missing some points including the cross-validation step. I try below to provide an overview of a common machine learning project. I assume a common project is a supervised machine learning problem (like iris dataset).

1. You defining the 'scope' or aim of the project :

• You have to define the purpose of the learning.

• When working with business in a company, it is a good idea to correctly express the need, the value of the project and its goal.

• You have to define the evaluation metrics (accuracy, recall, F1 score, AUC...). You can also define a minimum result you want to reach (say 80% accuracy for example).
• You can also ask yourself about the level of interpretability you need (do you care about model explanation? If no, maybe you could try more blackbox algorithms such as boosting, neural networks...).

2. Explore your data :

• Using statistics, visualization and intuition, try to learn your dataset and understand your features and labels.

• You can also search for missing data and outliers. Correct these observations refers as data cleaning process.

• Understanding your input variables will greatly help you to create and select relevant features.

3. Generating features/attributes from your own code/algorithm :

• This phase refers as features engineering. It is about creating features relevant to the learning problem.

• In this phase, you can clean your missing data and outliers in order to help the learning.

• You can derive new features from input variables relevant to your learning problem (handle categorical variables, rescale your features, apply transformations on input variables).

4. cross-validation :

• Cross-validation refers to your algorithm evaluation. In supervised machine learning, it is common, at least, to split dataset into 3 datasets (train, validation and test).

• Train dataset (about 60% of data) aims to train the algorithm.

• Validation dataset (20% of data) helps to find the best hyperparameters of your model (max depth for a tree, regularization for a linear/logistic regression...).

• Finally, test set (20% of data) gives you the true result you get on unseen data. It is the final evaluation.

5. Machine learning, feed the matrix into an algorithm :

• In this part, you train machine learning algorithms with regards to the cross-validation process (part 4).

• You can test different models. Some yield different performance results. Interpretability is not the same neither.

• To help the learning, you can diagnose your algorithm performs on both train and validation sets. This diagnostic is also called learning curves. It can tell you how to improve your learning. The purpose of learning curve is to help handle the underfitting/overfitting tradeoff. Underfitting is when you have a large bias error meaning your algorithm is not complex enough while overfitting means your algorithm is too complex and learns perfectly but is not able anymore to generalize learning on new unseen observations.

• You can also look at residuals (errors between predictions and real values) to improve your algorithm.

• Make features selection may also improve your algorithm learning.

6. Restitution

• Interpret the model and the performance you get.

• Create restitutions to business? Run into production?

Improve your machine learning model and performance is mostly about improving the above introduced points. By making new exploration, create new features, try a more powerful algorithm and so on, you can reach best results. Machine learning is a whole pipeline you have to optimize. I also think machine learning projects managements are really suitable with agile approaches.