How to choose PCA or KernelPCA a priori?

Question

I am learning about dimensionality reduction and I understood that one of the most used techniques in ML is PCA.

If I understood correctly, I use PCA whenever I want to reduce the number of features which should be mostly linearly separable (independent ?).

When the features are not-linearly separable (linearly not independent?), a nonlinear technique is required to reduce the dimensionality of a dataset and therefore I use KernelPCA.

Question: Supposing that what I just wrote is correct, how can I know in advance if the features are linearly or non-linearly separable before i decide which technique to use? So far the only way I was able to "guess" is by plotting the features after a PCA and checking if i can separate the new features through straight lines/surfaces. If I am not able to do so, then I apply Kernel PCA. Is this approach even correct?

Note:

1. Feel free to modify my question where the "?" are :)
2. My features don't have labels. It's an unsupervised problem.

Answer 1

Lots of the dimensionality problems are trial and errors at first.

Only very few datasets are linear "manifolds" that can be described with PCA. But it's a good start to see if there are disjoint sets, or if you can figure out some structure in 2D or 3D.

KPCA is a good technique, but there are many different kernels you can try. You may want to start with other "better" techniques (better as more deterministic) like ISOMAP, Laplacian Eigenmaps or LLE.

One of the things to remember is that you need enough data to populate the embedded space.