1
$\begingroup$

While I know it's possible to use StandardScaler on a SparseVector column, I wonder now if this is a valid transformation. My reason is that the output (most likely) will not be sparse. For example, if feature values are strictly positive, then all 0's in your input should transform to some negative value; thus you no longer have a sparse vector. So why is this allowed in Spark, and is it a bad idea to use StandardScaler when you want sparse features?

$\endgroup$

1 Answer 1

0
$\begingroup$

The default for the parameter withMean is False, so that data won't be centered, just scaled by the standard deviation(s).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.