PCA and K means in the context of machine learning

Question

I have a question regarding machine learning. I mostly do exploratory data analysis (EDA) of large 'omics datasets. I recently submitted a metabolomics based manuscript, and one of the reviewers was not happy with my analysis of the data. I think he as misunderstood the concept of machine learning, but I could be mistaken. I have added one of many examples where he refer to machine learning.

Reviewer: "PCA, K-means, and any other machine learning multivariate method of analysis must be performed on sex-specific groups"

I have always believed that machine learning is used for classifying and prediction purposes, but I dont seem to find any clear definition. Anyhow, I have three questions:

• Can somebody define machine learning?
• Is PCA machine learning? I have always thought its a method of dimension reduction that seeks to explain observed variance? It does not care about labels or groupings, nor does it predict or classify anything. Can somebody explain me where I got this wrong?
• Is K means clustering per definition machine learning? This algorithm has been around since the late 60s, long before machine learning was even a thing. I understand that this algorithm is applied in machine learning, but so is linear regression and logistic regression - but we dont call that machine learning.

I would appreciate your input on this?

Answer 1

It is always quite tricky to define Intelligence and Learning since they are very abstract concepts.

in my opinion, both algorithms are machine learning, and I will try to argue why.

Learning is (to me and Wikipedia) the action of improving at performing a task without being programmed explicitly. In our case, both algorithm improve at doing their task the more data we feed into them. And we do not code them explicitly, but instead we give them a goal : minimizing a loss function.

This is quite similar to the way humans learn. When we learn how to read, we get faster and faster and better at imagining what is written the more we read. Yet we do not know exactly how to read faster, we just know the objective function : minimize the time required to understand. And so our brain does the learning from its experiences (which are similar to the datas we feed to ML algorithms).

Another hint that leads to learning to me is the fact that both these algorithms won't give the same results twice if we initialize with random values. To compare to humans again, each person have his way of improving at the task : we all get better, yet each person has its own tricks for the task.

Anyway this is just my opinion on the subject and I do not pretend it does make sense to everyone.

Here is another interesting thread about PCA on Stack if you wanna read some more.

Hope my answer helps you in your philosophical reflection about this complicated topic.

Answer 2

Good and tricky questions! First of all, I am strongly against your point that machine learning is only used for classifying and prediction purposes. You might wonder why I disagree with your opinion: there is a general categorization towards ML: supervised ML and unsupervised ML. It seems that you mistake supervised ML as ML itself, while unsupervised ML whose target is to discover patterns and features from the dataset is neglected. Yes, PCA and K-Means won't help you foresee the output of the dataset, but you still manage to learn more about the dataset using them.

If you ask for the definition of ML, I would say ML is a data-based and data-triggered kind of AI, and AI is the computational simulation of human actions and intelligence.

Therefore, since PCA is a dimensionality reduction method conducted on the dataset, and K-Means is a clustering method, in other words, they reflect humans' desire to understand the data in easier way, their existence to some degree simulate people's manner to simplify the data and extract valuable information from the dataset. So again, PCA and K-Means is absolutely within the domain of ML.

You did make a point that PCA came into being before AI/ML did, but what you missed is that our theory also grows. The succeeding theory always has to satisfy the previous definition, while the predesessing category can be included by a larger domain. For example, does it make sense that I tell that root number two does not belong to irrational number set because it was found before irrational number was defined?