# Neural network: does bias equal to zero, is the same as, a layer without bias?

Question as in the title. Does bias equal to zero, is the same as, removing bias from the layer? Here's a pytorch implementation to showcase what I mean.

class MLP_without_bias(torch.nn.Module):
def __init__(self):
super().__init__()

# Bias set as False
self.linear = torch.nn.Linear(5, 3, bias = False)

# Xavier initialization
torch.nn.init.xavier_uniform_(self.linear.weight)

def forward(self, x):
return self.linear(x)

class MLP_with_bias_zero(torch.nn.Module):
def __init__(self):
super().__init__()

# Default bias set as True
self.linear = torch.nn.Linear(5, 3)

# Xavier initialization
torch.nn.init.xavier_uniform_(self.linear.weight)

# Bias initialized as zero
torch.nn.init.zeros_(self.linear.bias)

def forward(self, x):
return self.linear(x)


• In MLP_without_bias the bias will be zero after training, because of bias=False.
• In MLP_with_bias_zero the bias is zero at initialization, but this will not prevent it from being updated during training.