# Activation function vs If else statement

The question is very naive and most of us may know the answer. I have googled it but was not able to find a satisfactory answer so posting it here. Can someone please put the right words on this question.

Activation functions like ReLU, Sigmoid etc generally work like if else statements. e.g. if value of input is less than 0 then 0 else same value ( max(0,a) in case of ReLU) etc. So the question is why if else statements are not being used? Are these functions less compute intensive or work better in case of multidimensional data?

The argument in favor of if else statements is, the programming of neural networks is also being done in high level languages like python, C etc. In that case what compels the use of these activation functions rather than programmatic oriented If Else syntax?

• Well, ReLUs are implemented using if-else statements. – PascalIv Jan 7 '20 at 13:00

Activation functions in general cannot be implemented with if-else statements.

ReLU is a particularly simple activation function, and can thus easily be implemented with an if-else block. For example (in Python):

def relu(x):
return x if x > 0 else 0


But this only works because ReLU is linear for $$x \geq 0$$.

def approximate_logistic(x):