In multi-Linear regression where we have a model of $y=X_1+X_2$ it's a common practice (as I studied in my Master's) to increase the dimensionality and try to use a model of $y=X_1+X_2+X_1 X_2$. Can we say the same about increasing the nodes and the hidden layers in neural networks?
I'm a beginner and English is not my first language so sorry If I made a mistake or if it's a stupid question.
Any function $f(X)$ can be represented by a neural network of sufficient complexity, and as the size of the network is increased (number of nodes and/or number of layers), it can represent more complex functions. While you can think of this as similar to adding interaction terms in a linear model, neural networks are not limited to linear functions due to the use of non-linear activation functions. The other difference is that we don't control what terms or functions get included in the model as they are learnt during the training process.
