# Why is the input to an activation function a linear combination of the input features?

I'm new to deep learning, and there is something about the calculations I do not understand: For a standard neural network, why is it that only the activation function is not linear, but the input to the activation function is a linear combination of each of the $$x_i$$'s? For example, with the sigmoid function, it would look like: $$\frac{1}{1+ e^{-(w_0x_0 + w_1x_1 + b)}}$$ where $$w_i$$ are the weights and $$x_i$$ represents the input to that layer.

For example, why is it that we don't have something like this: $$\frac{1}{1+ e^{-(w_0x_0^2 + w_1\sqrt{x_1} + b)}}$$

Is it because it would be redundant if we had enough layers? Or is it because a priori, you wouldn't know what the best function is?

There's also the efficiency factor since a linear combination of $$n$$ inputs each having $$m$$ dimensions can be represented using a single matrix multiplication $$h=X \times W$$ where $$X$$ is an $$n \times m$$ matrix (where each row is an example and each column is a feature of that example) and $$W$$ is an $$m \times d$$ weight matrix. And computers are VERY efficient at doing matrix multiplications. Thus, the more you build your model to use matrix multiplications the better.