2
$\begingroup$

Say I have n dimensional data samples. I want to check the integrity of the features, if they are good representation of the respective classes, i.e. these features are good or not.

My plan is:

I use PCA and convert this to 2 dim data. Plot this data. See if they're separable enough.

Does the above plan sound okay for testing to see if features are any good?

$\endgroup$
4
  • $\begingroup$ Are you concerned with specific features or with the feature set as a whole? $\endgroup$
    – Hobbes
    Apr 11 '17 at 21:44
  • $\begingroup$ What do you mean by the "integrity" of the features? What do you mean by "good representation" of the classes? To check whether a set of features are good, train a classifier using all the features and see whether it can attain good accuracy. $\endgroup$
    – D.W.
    Apr 11 '17 at 23:42
  • $\begingroup$ @Hobbes both. How could I do it? $\endgroup$
    – user26763
    Apr 12 '17 at 0:32
  • $\begingroup$ yes in a way "good representation" might be it $\endgroup$
    – user26763
    Apr 12 '17 at 14:42
2
$\begingroup$

This was going to be a comment but it grew to an answer. I think there should be some clarification because the question itself is not specifically about visualization but checking the "integrity" of the features.

PCA will work for make a generalization of the dataset as a whole. It is a very standard starting point for exploring data. If the first 2 components do show a clear separation, this is a pretty solid indication that at least some projection of your data can well represent your classes. So, short answer is yes. If the first 2 components don't show separation, that does not mean that the features are necessarily bad, it just means that the first two components do not explain the majority of the variability in the dataset.

Feature selection is used to check the integrity/importance/value of individual features and how they affect the model. You can use random forest gini importance to rank your features or lasso regularization to with cross-validation to find out how individual features are weighted in a log. reg. model (this does require a little more work as the weightings are not necessarily an exact measurement of variable importance). Feature selection and cross-validation are the most direct ways of determining feature integrity. PCA is mostly a good first pass and helpful visualization.

$\endgroup$
2
  • $\begingroup$ Hi, thanks for your response, I'll try with PCA. Right now I'm in the process of selecting which features could be good $\endgroup$
    – user26763
    Apr 12 '17 at 19:17
  • $\begingroup$ Yup it's always good to try out with the simpler methods and if it doesn't work out then explore further by employing the other methods that others have mentioned. $\endgroup$
    – Impuls3H
    Apr 13 '17 at 0:43
3
$\begingroup$

I think t-SNE can be useful to visualize high dimensional data by 2D or 3D projection. As opposed to PCA, t-SNE learns a non-linear transformation which preserves the global data structure. Sklearn provides an easy to use implementation of t-SNE in here.

$\endgroup$
2
  • $\begingroup$ Hello thanks for responding, so my logic of converting high dim data into low dim to see if they're separable is correct? That is, if (features_NOT_separable_in_hidim_data) then after conversion to 2D, they should also be same (not differentiable)? $\endgroup$
    – user26763
    Apr 12 '17 at 14:40
  • $\begingroup$ t-SNE definitely has it's merits, but, it is computationally expensive, less intuitive than PCA, and the non-linearity makes it less reproducible. I think it's generally worth starting with simpler methods before jumping to more complex unless you really understand it. distill.pub/2016/misread-tsne $\endgroup$
    – Hobbes
    Apr 12 '17 at 14:46
3
$\begingroup$

By good I assume you're trying to see if the features are different enough? It depends on your data set actually. As smallchess has pointed out, PCA is commonly used and may work with your dataset. Try it out and see if the different classes are distinct? If not, you can try using higher dimensional visualisation tools.

Besides, with PCA you're reducing the dimensions to new features that may or may not be intuitive in meaning. That makes interpretation more complicated. If you're looking to visualise data, why not go with higher dimension visualisations? In case you're not aware of the different available tools.. Here is a question with great answers on Quora about higher dimension visualisation

Of the tools recommended, Andrews plot and parallel coordinates plots are very popular. I would suggest using these tools instead of using PCA and subsequently plotting for the purpose you're describing. If you have no need to visualise the data, sometimes you can just get a statistical summary/description of the datasets (mean, variance, min, max, etc). Below is an example of an Andrews Plot. You can plot the various classes into different colours to see if there's any discernible differences between the classes. Andrews Plot example on Matlab documentation

If that's not what you're looking for, to visualise the data to gauge the representation, just proceed to try out and build a model with the features and see if you get a good score?

Edit: Here are some shortcomings of PCA..if there's information that's misrepresented, do let me know! One thing to note is that with PCA, its transformation is orthogonal. Sometimes your data requires non-orthogonal representation. This is very dependent on how your data looks like. To give an example, take a look at the diagram below. Example of PCA shortcomings

In the above example, PCA fails to find PCs that maximises the variance due to orthogonality. Independent Factor Analysis may work better. Also, if you're planning to use the transformed data to build your model upon, there may be limitations with the linearly transformed data. Sometimes we need non-linear PCs. See the below example from the scikit webpage.

Scikit Kernel PCA

In such a case, a kernel PCA (utilizing a kernel trick similar in SVMs) may work better by getting a projection to get better linear distinction.

$\endgroup$
8
  • $\begingroup$ Thank you, dont know why you're getting downvoted though.. Yes I'm looking to see how features are different. So PCA wont work? $\endgroup$
    – user26763
    Apr 12 '17 at 1:04
  • $\begingroup$ It will depend on your dataset, if it can be reduced into a two dimensional feature space. You can get an estimate of that using a scree plot. For your purpose, I would reckon an Andrew's Plot would serve you better as it allows you to see which feature gives more information. About the downvote, I'm not too sure why, I'm no expert so did I explain anything incorrectly? $\endgroup$
    – Impuls3H
    Apr 12 '17 at 1:11
  • 2
    $\begingroup$ @user26763 I did the downvote because I believe the answer is not accurate. PCA in 2d doesn't necessarily mean "too much information loss". This is is just wrong. In fact, PCA plot on the 2D is very common and make total sense. You could label each PC with it's eigenvalue. The correct answer to your question is "PCA on 2D is common and makes total sense". $\endgroup$
    – SmallChess
    Apr 12 '17 at 1:37
  • $\begingroup$ You use the scree plot to estimate how many PC you want to maintain. But always almost just the first 2 PC for plotting and visualuzation (what the question is asking). If the features and data are distinct, they should be separable on the first two PC. -1. $\endgroup$
    – SmallChess
    Apr 12 '17 at 1:40
  • $\begingroup$ Ah I see. Thanks for clarifying smallchess. I didn't realise that I made such a sweeping statement on the information loss. A question if you don't mind, would using PCA and subsequently plotting a 2D or 3D scatter be better or using parallel plots or Andrews plot? Are there any particular cases that higher dimensional plots are better? $\endgroup$
    – Impuls3H
    Apr 12 '17 at 5:16
0
$\begingroup$

There are very good suggestions made above regarding the problem. In another approach, (somewhat similar but not same as PCA based menthods mentioned above) is to use an compressional autoencoder network and train it classwise (or one autoencoder). The compression can be from multi-dimension to 2 dimensions and can be easily plotted. Since the autoencoder(s) would learn to represent different classes, if the features selected for the original dataset can partition the data space adequately, then in a 2D scatter plot the differences could be made out.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.