# Is it usual for Scikit learn's standard scaler to cause non-invertibility?

For example, I am trying to perform linear regression on the following set of data

Data examples: $$X = [[1, 20], [3, 40], [5, 60]]$$ (each row is an example, there are three examples, each with a feature of $$2$$, arranged in Numpy array)

Targets: $$y = [1, 2, 3]$$ (whatever you like, it doesn't affect our result.

Fitting a standardscaler gives me,

X = [[1, 20], [3, 40], [5, 60]]
scaler = StandardScaler()
scaler.fit(X)
Y = scaler.transform(X)


$$Y = [[-1.22474487 -1.22474487] [ 0. 0. ] [ 1.22474487 1.22474487]]$$

Now I want to compute the normal equation of a linear regression problem. This inolves calculating the following matrix $$Z = (Y^T Y)^{-1} Y^T$$

Z = np.linalg.inv(np.dot(np.transpose(Y), Y))*np.transpose(Y)


I get LinAlgError: Singular matrix

Note that this does not seem to be a problem with the original data set $$X$$

Is this a usual behavior or did I do something wrong?

• Since you have two features, YtY should be a 2×2 matrix and not 3×3 matrix. Try (YYt)inv Y instead. Jun 4 at 5:14

The matrix $$X$$ has full rank: the columns demonstrate an affine relationship ($$x_2=10x_1+10$$), but not a linear one. So $$X^T X$$ (which is $$2\times2$$) is indeed invertible, and everything proceeds normally.
If you add an all-ones column to $$X$$ (to incorporate an intercept to the OLS), you elevate the affine relationship to a linear one, and you'll find that $$X^T X$$ is not invertible.
The StandardScaler (in addition to scaling) centers the features, which again rips away the bias/shift, and turns the affine relationship to a linear one (of course, it's the identity relationship).
A linear or affine (with fixed constant term) transformation like scaling should not change the rank of a matrix nor its invertibility. This is one point. However according to documentation of StandardScaler, StandardScaler scales each feature independantly (ie. no longer a uniform linear/affine transformation, constant terms are multiple), so this can in fact alter the rank and invertibility of a matrix of data (for example use a different scaler).