# Duplicated features for gradient descent

Suppose that our data matrix X has a duplicated column, i.e, there is a duplicated feature and the matrix is not full column rank. What happpens?

I guess that we can not find a unique solution because that's the case for the close form in linear regression, but I do not see how to show that intuitively, or even if it is true or not.

• In what context? What kinds of model? – Ben Reiniger Jan 26 at 18:23
• Linear regression – vicase98 Jan 26 at 18:39

Duplicated data is a special case of linear dependence, which is a special case of correlation. Say you have duplicated variables $$X1 = X2$$, so the network output is constant over the weight lines $$w_2 = c - w_1$$, where c is a constant. It means that the gradient of the error is 0 along those lines : moving along those lines has no effect on the learning. In that sense the solution won't be unique.