Suppose that we are training a linear regressor (perceptron). Adding extra features that are not related to the target (e.g. randomly generated values) before training will typically ____ our training error.
Would it typically increase, decrease or stay the same?
Listed Answer: decrease
The answer to this question is listed as decrease, but wouldn't adding extra unrelated features not change the training error, because the model would just learn to ignore those features (add 0 weights to them). Even if the model didn't learn to ignore the extra features, if anything wouldn't the training error increase as it doesn't seem like the model would be able to learn a relationship that would benefit the training error from a completely random unrelated feature?