During feature engineering, we can create new features out of existing ones by using arithmetic operations albeit linear or not.
Let's say we have two features x and z. We can then create (engineer) a new feature f by summing x & z, assuming this makes sense in the context of the use case, to therefore become, f = x + z.
Or if a non linear feature is to be created then something like the following can be implemented, f = x*x + z.
My question is, given that we have x & z and knowing the strength of their correlation to the target variable, what is the point of creating a third feature which is just a combination of the original two?
What can the third feature point to that the first two can't?
Maybe if the combination is non-linear, I can understand, but what if the combination is linear? Why should it help?
P.S I have stumbled on a post on Cross-Validated addressing this issue, but given the nature of the answer, it still left me wondering and unsatisfied.