# Weights initialization in Neural Network

I was viewing code for custom neural network for sentiment analysis. It had 3 layers (1 hidden layer). I am more concerned with weight initialization for the layers

 self.weights_0_1 = np.zeros((self.input_nodes,self.hidden_nodes))
self.weights_1_2 = np.random.normal(0.0, self.output_nodes**-0.5,(self.hidden_nodes,self.output_nodes))


What is the idea behind initializing zero's weight matrix. I have learned that initializing weights to zero might lead to linearity.

This might be a very vague question, I will be happy to provide any specifics you want. https://github.com/udacity/deep-learning/blob/master/sentiment-network/Sentiment_Classification_Solutions.ipynb

• Please provide the source, because as you already pointed out this is generally a bad idea. Dec 23 '18 at 11:07

A single layer that gets initialized with only 0 will not converge, because the derivatives will be strictly identical (this rule also applies for any such last layer in a network, the layer gets basically useless). Here, there is a second layer after that saves the day, but it's still a bad practice in general.