# Assessing Group Similarities and Dissimilarities Post PCA

The goal is to assess similarity and dissimilarity between 6 known groups.

The original data began with the 6 known groups and 2,700+ variables all on a scale of 0 to 100.

I have performed PCA to reduce the 2700+ variables into 5 principal components using the dudi.pca function from the ade4 package in R. Here are the Eigenvalues for the components:

      eigenvalue variance.percent cumulative.variance.percent
Dim.1   998.3274        36.635867                    36.63587
Dim.2   670.1278        24.591848                    61.22771
Dim.3   482.2372        17.696776                    78.92449
Dim.4   352.2806        12.927728                    91.85222
Dim.5   222.0270         8.147781                   100.00000


I would now like to assess the distances between the 6 known groups. Is this done as simply as generating a distance matrix using each group's coordinates for each of the principal components? If so, I am leaning towards using Manhattan distance to get the absolute distance.

Here are the coordinates of each group:

           Dim.1       Dim.2       Dim.3        Dim.4        Dim.5
Group 1    69.019038    7.940190    0.4985599  - 6.847178     0.3964117
Group 2   -16.302322  -25.965373  -29.3084201  -23.013430     9.9183010
Group 3   -26.313850   50.159662    6.9486408  -10.713924     5.2883152
Group 4   -12.800767  -26.211432   39.5067264  - 8.775551   - 8.8840592
Group 5   - 9.228404    2.648632  -20.4297314   16.685426   -26.8559444
Group 6   - 4.373694   -8.571679    2.7842244   32.664657    20.1369757


If not, what would be the appropriate way to assess individual similarity/ dissimilarity post PCA?

• I think you need to give a little more context into the problem (e.g. what question you're trying to answer) – David Marx Feb 13 '18 at 8:16
• Thanks David, the question to be answered is how different are the 6 known groups in terms of their affinities towards certain digital behaviours/interests. Each group has an affinity to the 2,700 digital behaviours on a scale of 0 to 100, so based on those behaviour affinities, how different is each group compared to eachother. Hope that clarifies – hc_ds Feb 13 '18 at 8:21
• What are you hoping to learn? A hierarchy of groups? A suggested reduction into a smaller number of groups? A profile for a group with respect to its most discriminating behaviors? Which group is the most unlike the others? Having learned "how different each group is", what are you planning to do with that information? Concretely, what are you trying to learn? What's your hypothesis? – David Marx Feb 13 '18 at 8:30
• I want to learning the following: 1) what are the drivers of difference between groups. 2) which groups are the most similar and which are the most different. And this is info is needed to drive decisions on how to interact with each group – hc_ds Feb 13 '18 at 8:42
• Regarding (1): do you want to know how each of your features (the digital behaviors) influences group membership? Just the most important? Do you need to be able to rank features somehow? Regarding (2): why is it helpful to understand which groups are the most similar/different? Do you need to be able to understand this similarity/difference in the context of the drivers? What does similarity/difference mean to you in a business context? How does knowledge of group similarity/difference affect your decision process? – David Marx Feb 13 '18 at 8:58

Assuming you pick e.g. Euclidean distance, one simple way to find inter-group distance would be to first calculate the group centroid, i.e. average location of each group by averaging the locations of each group together. You can do this by just averaging all the 5x1 vectors belonging to a group, and repeat for each group. Then calculate the Euclidean distance between the resulting 6 centroids, giving you a 6x6 distance matrix.

• Thanks @tom. The distance indicates how similar or dissimilar the individual groups are when it comes to the interests making up each of the components – hc_ds Nov 15 '17 at 20:44

To measure what features are the "drivers of difference between groups", you're going to need to frame this as a classification problem. Within this framework, you can use variable importance (and potentially coefficient values, depending on the model) to perform inference on drivers, i.e. to identify and rank them.

Applying PCA as a pre-processing step will make it significantly harder to identify what variables are drivers for specific classes. PCA is agnostic to class assignment: you can use PCA eigenvector components and loadings to interpret what variables are responsible for the bulk of the variance in your data, but that isn't actually what you want. Imagine if your classes were long elipsoidal clusters, each with the exact same covariance matrix but a different center (i.e. the elipses are "parellel"): the components given by PCA would be mainly influenced by the elipsoid shape (i.e. the shared covariance matrix) rather than by differences between groups. If you're interested in understanding the drivers of group differences, I strongly recommend you drop the PCA step.

I still don't have a good handle on exactly what you're hoping to get out of "which groups are most similar", but I suspect the "proximity" measure given by random forests would satisfy your need here. This measure is actually between observations, but you can take averages to get the expected proximity between groups. A benefit of using random forests here is that they have built-in variable importance measures, and you can even introspect them to understand the drivers behind individual observations.

Here's a little demo showing how to use a random forest model to detect drivers via variable importance, and measure group similarity via average inter-observation proximity:

First, set up the data and fit a model

library(randomForest)
data(iris)
set.seed(123)

unq_classes <- unique(iris$Species) n_classes <- length(unq_classes) iris.rf <- randomForest(Species ~ ., data=iris, importance=TRUE, proximity=TRUE)  Variable importances, rescaled to [0,1] with "1" indicating the most important variable: var_imp <- importance(iris.rf)[,1:(n_classes+1)] var_imp <- apply(var_imp, 2, function(m) m/max(m))  Here's the result (that last column is the marginalized importance):  setosa versicolor virginica MeanDecreaseAccuracy Sepal.Length 0.2645863 0.2403256 0.2503047 0.3336813 Sepal.Width 0.1927240 0.0314708 0.1716495 0.1564093 Petal.Length 0.9525359 0.9589636 0.9356667 0.9549433 Petal.Width 1.0000000 1.0000000 1.0000000 1.0000000  And our mean proximities, again rescaled to [0,1] with 1 indicating the most similar pair of groups: prx <- iris.rf$proximity
mean_prx <- matrix(NA, n_classes, n_classes)
for (i in 1:(n_classes-1)){
for (j in (i+1):n_classes){
cls_i <- iris$Species == unq_classes[i] cls_j <- iris$Species == unq_classes[j]
mean_prx[j,i] <- mean(prx[cls_i, cls_j])
}
}

mean_prx <- mean_prx/max(mean_prx, na.rm=TRUE)
rownames(mean_prx) <- unq_classes
colnames(mean_prx) <- unq_classes


Giving us:

                 setosa versicolor virginica
setosa               NA         NA        NA
versicolor 0.0267520374         NA        NA
virginica  0.0007778552          1        NA


Here's what the data looks like to put these results in context: