Skip to main content
Data Science

Return to Answer

Corrected Understanding
Source Link
Ronak Agrawal
Ronak Agrawal
  • 206
  • 3
  • 11

This is the summary of what I have researched so far:

Fundamentally:

Model: A model is a set of rules that fit/represent the trends/rules in data supplied.

Overfitting: Overfitting in general sense is modeling of noise/randomness rather thanalong with the sample where the effect of noise affects model result.

With the fundamentals at hand, one can have an intuition that, WHEN YOU FIT, THERE IS A CHANCE THAT YOU CAN OVERFIT. i.e. When you can model something that is REQUIRED, there is a good chance you can model something which is NOT REQUIRED.

So, YES, OVERFITTING IS POSSIBLE IN UNSUPERVISED LEARNING.

If PCA can be used to remove/reduce overfitting in Unsupervised Learning?

Supervised learning uses labels as a comparison measure, i.e. 2 samples are compared in data(feature sets, feature vectors or whatever jargon one throws in here.), w.r.t. their labels to identify patterns. So, PCA is a technique which does not consider Labels. So, removal of data with PCA is not preferred for supervised, as it may remove data for which feature may not have enough information but labels do.

So PCA is not recommended for removing Overfitting for Supervised Learning. You can use it, if at all, with the risk that you may lose information from your data.

Unsupervised learning does not have labels, instead, it inter-compares 2 samples to identify patterns.

Basically, there is NO data for which feature may not have enough information but labels do as labels don't exist. So, PCA will help you reduce dimensionality as it would tend to defer data that doesn't add much information.

Having said above, it is NOT necessary that it would definitely help you reduce overfitting.

But, it's worth a try. As, if the noise is dominant in the data, there is a definite pattern in the data, and your model is just abstracting it which drills down to modeling your data.

So yes, PCA can help you reduce overfitting in your data, and to the question that is it a good practice?

I haven't come across an article or reasoning that defers it for Unsupervised. Regardless of everything, PCA does seem a practical approach to reducing Overfitting for Unsupervised Learning without loss of information.

This is the summary of what I have researched so far:

Fundamentally:

Model: A model is a set of rules that fit/represent the trends/rules in data supplied.

Overfitting: Overfitting in general sense is modeling of noise/randomness rather than the sample.

With the fundamentals at hand, one can have an intuition that, WHEN YOU FIT, THERE IS A CHANCE THAT YOU CAN OVERFIT. i.e. When you can model something that is REQUIRED, there is a good chance you can model something which is NOT REQUIRED.

So, YES, OVERFITTING IS POSSIBLE IN UNSUPERVISED LEARNING.

If PCA can be used to remove/reduce overfitting in Unsupervised Learning?

Supervised learning uses labels as a comparison measure, i.e. 2 samples are compared in data(feature sets, feature vectors or whatever jargon one throws in here.), w.r.t. their labels to identify patterns. So, PCA is a technique which does not consider Labels. So, removal of data with PCA is not preferred for supervised, as it may remove data for which feature may not have enough information but labels do.

So PCA is not recommended for removing Overfitting for Supervised Learning. You can use it, if at all, with the risk that you may lose information from your data.

Unsupervised learning does not have labels, instead, it inter-compares 2 samples to identify patterns.

Basically, there is NO data for which feature may not have enough information but labels do as labels don't exist. So, PCA will help you reduce dimensionality as it would tend to defer data that doesn't add much information.

Having said above, it is NOT necessary that it would definitely help you reduce overfitting.

But, it's worth a try. As, if the noise is dominant in the data, there is a definite pattern in the data, and your model is just abstracting it which drills down to modeling your data.

So yes, PCA can help you reduce overfitting in your data, and to the question that is it a good practice?

I haven't come across an article or reasoning that defers it for Unsupervised. Regardless of everything, PCA does seem a practical approach to reducing Overfitting for Unsupervised Learning without loss of information.

This is the summary of what I have researched so far:

Fundamentally:

Model: A model is a set of rules that fit/represent the trends/rules in data supplied.

Overfitting: Overfitting in general sense is modeling of noise/randomness along with the sample where the effect of noise affects model result.

With the fundamentals at hand, one can have an intuition that, WHEN YOU FIT, THERE IS A CHANCE THAT YOU CAN OVERFIT. i.e. When you can model something that is REQUIRED, there is a good chance you can model something which is NOT REQUIRED.

So, YES, OVERFITTING IS POSSIBLE IN UNSUPERVISED LEARNING.

If PCA can be used to remove/reduce overfitting in Unsupervised Learning?

Supervised learning uses labels as a comparison measure, i.e. 2 samples are compared in data(feature sets, feature vectors or whatever jargon one throws in here.), w.r.t. their labels to identify patterns. So, PCA is a technique which does not consider Labels. So, removal of data with PCA is not preferred for supervised, as it may remove data for which feature may not have enough information but labels do.

So PCA is not recommended for removing Overfitting for Supervised Learning. You can use it, if at all, with the risk that you may lose information from your data.

Unsupervised learning does not have labels, instead, it inter-compares 2 samples to identify patterns.

Basically, there is NO data for which feature may not have enough information but labels do as labels don't exist. So, PCA will help you reduce dimensionality as it would tend to defer data that doesn't add much information.

Having said above, it is NOT necessary that it would definitely help you reduce overfitting.

But, it's worth a try. As, if the noise is dominant in the data, there is a definite pattern in the data, and your model is just abstracting it which drills down to modeling your data.

So yes, PCA can help you reduce overfitting in your data, and to the question that is it a good practice?

I haven't come across an article or reasoning that defers it for Unsupervised. Regardless of everything, PCA does seem a practical approach to reducing Overfitting for Unsupervised Learning without loss of information.

Source Link
Ronak Agrawal
Ronak Agrawal
  • 206
  • 3
  • 11

This is the summary of what I have researched so far:

Fundamentally:

Model: A model is a set of rules that fit/represent the trends/rules in data supplied.

Overfitting: Overfitting in general sense is modeling of noise/randomness rather than the sample.

With the fundamentals at hand, one can have an intuition that, WHEN YOU FIT, THERE IS A CHANCE THAT YOU CAN OVERFIT. i.e. When you can model something that is REQUIRED, there is a good chance you can model something which is NOT REQUIRED.

So, YES, OVERFITTING IS POSSIBLE IN UNSUPERVISED LEARNING.

If PCA can be used to remove/reduce overfitting in Unsupervised Learning?

Supervised learning uses labels as a comparison measure, i.e. 2 samples are compared in data(feature sets, feature vectors or whatever jargon one throws in here.), w.r.t. their labels to identify patterns. So, PCA is a technique which does not consider Labels. So, removal of data with PCA is not preferred for supervised, as it may remove data for which feature may not have enough information but labels do.

So PCA is not recommended for removing Overfitting for Supervised Learning. You can use it, if at all, with the risk that you may lose information from your data.

Unsupervised learning does not have labels, instead, it inter-compares 2 samples to identify patterns.

Basically, there is NO data for which feature may not have enough information but labels do as labels don't exist. So, PCA will help you reduce dimensionality as it would tend to defer data that doesn't add much information.

Having said above, it is NOT necessary that it would definitely help you reduce overfitting.

But, it's worth a try. As, if the noise is dominant in the data, there is a definite pattern in the data, and your model is just abstracting it which drills down to modeling your data.

So yes, PCA can help you reduce overfitting in your data, and to the question that is it a good practice?

I haven't come across an article or reasoning that defers it for Unsupervised. Regardless of everything, PCA does seem a practical approach to reducing Overfitting for Unsupervised Learning without loss of information.