# Counting the number of layers in a neural network

I am going over the Udacity tutorial on Neural Networks.

Here's a diagram from the tutorial:

What makes this a '2 layer neural network'?

• I was under the impression that the first layer, the actual input, should be considered a layer and included in the count.

• This screenshot shows 2 matrix multiplies and 1 layer of ReLu's. To me this looks like 3 layers. There are arrows pointing from one to another, indicating they are separate. Include the input layer, and this looks like a 4 layer NN.

Input layer is a layer, it's not wrong to say that.

However, when calculating the depth of a deep neural network, we only consider the layers that have tunable weights.

• So in the picture above that I posted, I see 2 matrix multiplies and a ReLu. Isn't that 3 layers? – Monica Heddneck Sep 16 '16 at 5:39
• ReLU is just element-wise activation, not a weighted layer. – nn0p Sep 16 '16 at 7:07
• @MonicaHeddneck nn0p is correct. Don't be too concern what's exactly drawn in the picture. Focus on the actual layer architecture (the text below). – SmallChess Sep 16 '16 at 7:18
• ohhhhh so ReLu's can be layers because they are not parametric! This is exactly what I was wondering about. – Monica Heddneck Sep 16 '16 at 7:18
• @MonicaHeddneck ReLus is part of the hidden layer. The picture you show is misleading. – SmallChess Sep 16 '16 at 7:21

This is a 2-layer network because it has a single hidden layer and an output layer. We don't count the first layer. When we say 3 layers, we actually mean 2 hidden layers and an output layer. Perhaps this helps you?

EDIT: We don't count the input layer because there's no parameter (bias + weights). In actual implementation, it's not implemented. Netural network framework simply "connects" the input features to the first adjustable layer (eg: hidden layer).

• That makes a lot of sense. Can you take a look at the image I posted -- I think I'm seeing 3 layers that all are parameterized. – Monica Heddneck Sep 16 '16 at 5:49
• @MonicaHeddneck I don't know what exactly the picture is drawing, but it's not important. The text below tells you exactly why it's a two-layer network. – SmallChess Sep 16 '16 at 5:59
• I agree now...the text is correct and the picture is misleading. When I asked this question, I wasn't sure if the picture was correct and the text was misleading. Thanks for your help. – Monica Heddneck Sep 16 '16 at 7:28
• @SmallChess Hi, can you please give the URL of the article/webpage from which you imported the picture containing the two NN diagrams and the Naming convertions: paragraph ? – SebMa Jul 20 '18 at 13:55
• @SebMa I googled partial of the text, luckily google is smart enough :) cs231n.github.io/neural-networks-1 – Yana Agun Siswanto Jul 10 '19 at 23:31