1
$\begingroup$

I am trying to build a classifier to predict the ratings of a show during a specific time.

I have extracted around 109 features, some relating to the time field namely,

  • Day of Year
  • Month of year
  • Is on a weekend?
  • During business hours?
  • Public Holiday?

I also included some categorical features and used a label binariser for which channel it appeared on, and the broadcaster.

I wanted to check the linearity of the dataset, which would inform me as to whether a linear regressor could be used or something non-linear like a neural network. I decided to do dimensionality reduction using PCA in order to visualise if the dataset was linearly separable in 2D.

from sklearn.decomposition import PCA
pca = PCA(n_components=2)
data_scaled = pca.fit_transform(df[cols])
plt.plot(data_scaled[:,0], data_scaled[:,1], 'ro')
plt.xlabel('first component')
plt.ylabel('second component')
plt.show()

PCA results

I am very confused by the result and am not able to interpret this.

The plot of the first component:

Plot of the first component

The plot of the second component:

enter image description here

What could the PCA results tell? What would cause these graphs?

$\endgroup$
2
$\begingroup$

Do not apply PCA to categorical data

PCA attempts to find the dimensions which contains the most variance in a dataset. When you have categorical variables, the distance between points and the variance captured by a variable are ill-defined. First of all, you have no proper distance measure to tell how far apart two categories are, and secondly, the ordering of your categories will have a big impact. Ordering months categorically will result in a small distance between January and February, but a large one between December and January. If you're categorizing things like broadcaster or channel, there is no natural order to put things in, so your distance measure will be essentially meaningless. Your PCA plot looks strange because you have applied it to categorical data - this is not a meaningful representation of your data.

$\endgroup$
1
$\begingroup$

Each PCA component is a projection of your centered data onto a line. Centering puts your origin into the center of the multidimensional data. Then each component line direction is chosen so that the projection has the greatest variability. Successive component directions are orthogonal by construction.

Generally, PCA is intended for continuous data (as opposed to categorical). It can also work with categorical data when there are many categories and the categories are ordered (i.e. approaching continuous data).

What you are seeing is the categorical nature of your data. Your first component is the 12 months and your second component is the 7 days of the week. It appears that these two variables have the largest variance in your data. They are equally spaced because their numerical representation is equally spaced.

Possibly the best book on PCA is http://cda.psych.uiuc.edu/statistical_learning_course/Jolliffe%20I.%20Principal%20Component%20Analysis%20(2ed.,%20Springer,%202002)(518s)MVsa.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.