# What what will happen if all the layers of a MLP or any DL architecture are set as same in the beginning?

Setting the initial weights as all zeros will have the output dependent on the bias and setting the weights of all the neurons of a layer as same, will update the gradients in same way thus removing the effect of non-linearity and under fitting of model.

Now I want to ask that if a model has 3 layers (of course with same number of neurons) and we initialise all the weights and biases of the network as same, given no two neurons in the same layer has same values. For example, let us suppose that weights of all the W in all the layer at the beginning is as:

[[0.1,0.2,0.3],
[0.4,0.5,0.6]]


Now what will happen?

I guess the model will be under fitting and will be linear in nature because in case we use ReLu, the output will be just the scaled version of the prevision layer?

• Initialization of weights in NNs can have various effects which most of the time can be safely to ignored in practice. Terms such as "underfitting" are not justifiable so plainly Jun 27 at 13:28