1
$\begingroup$

I have a regression problem whereby my data has 21 features and I wish to apply dimensionality reduction using PCA. As far as I know, all the tutorials I have seen so far use PCA for classification problems. I did do PCA for regression but I am un-able to display the nice scatterplots that show the PC1 on the x-axis, the PC2 on the y-axis, and the targets in the middle.

I wrote the following code

        X = self.X

        pca = PCA(n_components=NUM_FEATURES_PCA)
        principal_components = pca.fit_transform(X)

        principalDf = pd.DataFrame(data=principal_components,
                                   columns=['PC1', 'PC2'])
        finalDf = pd.concat([principalDf, self.df[[self.target_variable]]], axis=1)
        plt.scatter(finalDf.loc[self.df[self.target_variable], 'PC1']
                   , finalDf.loc[self.df[self.target_variable], 'PC2'], s=50)
        plt.xlabel('PC1', fontsize=15)
        plt.ylabel('PC2', fontsize=15)
        plt.title('2 component PCA', fontsize=20)
        plt.show()

So in other terms, could we display such plots for PCA in regression ? Or should we transform the continuous target variable to a categorical (labeled one) through binning or like-wise ?

references: these plots

$\endgroup$
1
$\begingroup$

First of all, you can project your explanatory variable (continuous) in your first plane (PC1 + PC2). The direction of the arrow (projection) and how far goes from the axis origin will tell you how the points are distributed according to this representation of variables in your factorial plane.

On the other hand, the quick answer is to group your continuous variables into chunks (descretizise your variable into an ordinal one), then you'll have the same plot as the reference.

Furthermore, you could try to color your scatter plot using a color scale (from white to black, red to blue...), and you'll see if there's some kind of progression of your data in that factorial plane according to the continuous variable.

These three "strategies" actually are showing the same, although the second one is more sensitive to the cuts.

Summarizing:

  1. Project the continuous variable in the factorial plane (what PCA usually does).
  2. Group your continuous variable in groups and plot it (easier way to see although it is sensitive).
  3. Color your scatter plot using an scale (using min and max value of your continuous variable).
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.