# Why are eigenvectors produced by np.linalg.eig different than the PCA components stored in the instance of the PCA object?

I am trying to understand why eVec (produced by np.linalg.eig) is different than pca.components_.T from the instance of the PCA class. It was my understanding that the eigenvecters of the covariance matrix are the principal components after descending sort by eigenvalues.

An explanation in simple terms would be appreciated.

import pandas as pd
import numpy as np
from sklearn.decomposition import PCA
from sklearn import preprocessing
from sklearn.preprocessing import StandardScaler

df = pd.read_csv(
'https://gist.githubusercontent.com/seankross/a412dfbd88b3db70b74b/raw/5f23f993cd87c283ce766e7ac6b329ee7cc2e1d1/mtcars.csv'
)

df = df.drop(['model', 'vs', 'am'], axis = 1)

df = df.apply(lambda x: pd.to_numeric(x))

M = df.to_numpy()

Mnorm = M-np.mean(M, axis=0)

Mnorm = Mnorm/np.std(M, axis=0)
#  This is the normalized source data.

C = (Mnorm.T @ Mnorm) / (Mnorm.shape - 1)
#  This is the Covariance Matrix without bias.

eVal1, eVec1 = np.linalg.eig(C)

eVal = eVal1[np.flip(np.argsort(eVal1))]
#  eVal is sorted according to the order of the eigenvalues.

eVec = eVec1[np.flip(np.argsort(eVal1))]
#  The same sort order as above is applied to the eigenvectors.

### From sklearn:

scaler = StandardScaler()

scaler = scaler.fit(df.to_numpy())

Anorm = scaler.transform(df.to_numpy())

pca = PCA(n_components=9)

pca_transform = pca.fit_transform(Anorm)

assert (Mnorm == Anorm).all().all()
#  This tests that Mnorm was probably constructed correctly.

assert (C.round(10) == pca.get_covariance().round(10)).all().all()
#  This indicates that the Covariance Matrix (C) was constructed correctly - the rounding is arbitrary.

assert (eVec.round(5) == pca.components_.T.round(5)).all().all()
#  However, eVec and pca.components_.T are not equal.


## 1 Answer

Two problems:

1. Your sorting is incorrect:
eVec = eVec1[np.flip(np.argsort(eVal1))]


sorts the rows of the matrix, but you want to sort the columns. Replacing this with

eVec = eVec1[:, np.flip(np.argsort(eVal1))]


fixes this issue.

1. The sign of the eigenvectors are sometimes opposite. (That's fine, being an eigenvector is scale invariant, and while both np.linalg and sklearn use unit eigenvectors, that still leaves a choice of two. I'm not sure how the packages end up picking one.)