I implemented a multi-class classification and wanted to test it using the MNIST dataset. I realized that if I use standardization
$X \leftarrow \frac{X-mean(X)}{std(X)}$,
over 50% of all features will be zero. Is that a problem?
Does it make more sense to work in such a case with normalization
$X \leftarrow \frac{2(X-min(X))}{max(X) - min(X)} - 1$,
such that all features are between -1 and 1?
What about doing first a standardization followed by a normalization step?
This is just vocabulary, but standardization is a specific normalization. So you're comparing two different normalizations.
With standardization, the mean feature will be zero. If the distribution of the feature is symmetric, like a normal distribution with a mean of 100, or whatever, then after standardization, 2/3 of the values for this feature will be less than 1.
Now about the min-max normalization. Imagine your raw feature values run from 0 to 10 but are almost always 0. Then after this normalization, the value of this feature will be -1 for most examples (corresponding to the min value).
None of this is likely to be a problem, although in principle it depends on the algorithm that has to make sense of these inputs.
Welcome! Sounds like your data set is highly imbalanced. For techniques on handling an imbalanced data set, please have a look at this. https://machinelearningmastery.com/tactics-to-combat-imbalanced-classes-in-your-machine-learning-dataset/
EDIT: Looks like I misunderstood the question, sorry! Also: I think the term "multi-class regression" does not exist. It sounds more like you're doing multi-class classification. If your input features are consistently 0, then it sounds like your actual value of each feature and the mean of the feature are identical, in which case you have very poor features. The features will not do a good job of differentiating between the classes. You should try (or hopefully use a neural network to learn) a new set of features.
