The question is very naive and most of us may know the answer. I have googled it but was not able to find a satisfactory answer so posting it here. Can someone please put the right words on this question.
Activation functions like ReLU, Sigmoid etc generally work like if else statements. e.g. if value of input is less than 0 then 0 else same value ( max(0,a) in case of ReLU) etc. So the question is why if else statements are not being used? Are these functions less compute intensive or work better in case of multidimensional data?
The argument in favor of if else statements is, the programming of neural networks is also being done in high level languages like python, C etc. In that case what compels the use of these activation functions rather than programmatic oriented If Else syntax?