1
$\begingroup$

so, let's say I have a set of 3D points. Let's say these points lie more or less on a plane that is embedded in the 3d space, then I can use PCA to 'compress' these 3D points to 2D coordinates on that plane, such that they still aproximate the original data well.

let's say half of the 3d points don't lie close to that plane, but instead close to some other plane.

If I just do PCA and reduce to 2 dimensions, I won't get a good aproximation.

If the algorithm however would 'see' that some of the 3d points compress well onto one plane, and others compress well on another plane and label each point and do PCA separately for each set (and compress them to points with 2 coordinates plus one bit that says which set it belongs to) it will aproximate the original data much better.

What's the name for such a PCA algorithm that is also capable of splitting the input data into maximally N sets (probably with some penalty on the number of sets), such that for each set dimensionality reduction yields a much better fitting than if all data points would be reduced together?

// Edit: adding an example. If one would only cluster by distance in the high-dim space one would arrive at the bad clustering where there are more clusters and each cluster would have a higher error when projected down.

the good example uses fewer clusters and they project better on their 2 dimensional sub-spaces (the green cluster being able to even compress to a 1D space)

enter image description here

$\endgroup$
  • 1
    $\begingroup$ "If I just do PCA and reduce to 2 dimensions, I won't get a good aproximation." this is not so, you will get two different clusters of 2d points $\endgroup$ – Nikos M. Jul 29 at 19:25
  • 1
    $\begingroup$ its called clustering (by some clustring algorithm) the features from PCA $\endgroup$ – Nikos M. Jul 29 at 19:26
  • $\begingroup$ Technically this doesn't really reduce the dimensionality since you still have 3 dimensions at the end, it only makes the 3rd dimension a binary categorical feature representing whether the point belongs to cluster 1 or 2. $\endgroup$ – Erwan Jul 29 at 22:50
  • $\begingroup$ @Erwan in this example yes, but let's say you reduce from 1000 dimensions to 50 dimensions with 10 different clusters, then you have 51 dimensional points (of which one dimension is an index into the 10 clusters) $\endgroup$ – user1282931 Jul 29 at 23:05
  • $\begingroup$ @NikosM. how? note that I do not know which points belong to which cluster initially (or how many clusters there are at all for which it makes sense to cluster them) $\endgroup$ – user1282931 Jul 29 at 23:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.