# What is fully connected layer additive bias?

I'm going to use PyTorch specifically but I suspect my question applies to deep learning & CNNs in general therefore I choose to post it here.

Starting at this point in this video and subsequently:

George H. explains that the PyTorch function torch.nn.Linear with the bias parameter set to False then makes the torch.nn.Linear functionally equivalent (other than GPU support of course) to the following NumPy line:

x = np.dot(weights, x) + biases


Note that in torch.nn.Linear bias by default is set to True:

https://pytorch.org/docs/stable/generated/torch.nn.Linear.html

Here is the PyTorch documentation for the bias parameter:

bias – If set to False, the layer will not learn an additive bias. Default: True

Can anybody please explain what "additive bias" is? In other words, what additional steps is PyTorch doing if torch.nn.Linear bias parameter set to True? Surprisingly I was not able to find much on this topic upon Googleing.

You probably misanderstood the video, it is not said that a linear layer with Bias set to False is equivalent to :

x = np.dot(weights, x) + biases


Because that is not true, a layer without Bias is equivalent to

x = np.dot(weights, x)


The way he recreates the layer without Bias is actually with the following function :

x = x.dot(l1)  # X = W1.X  First linear layer
x = np.maximum(x, 0)  # X = ReLU(X)
x = x.dot(l2)  # X = W2.X  Second linear layer


Setting Bias=True means the layer has the second bias term it adds after multiplying the weights with the input (as in the formula you quoted) :

x = np.dot(weights, x) + biases