I'm looking through an online course about machine learning and the first big topic is finding a model that approximates our data with linear regression. The model itself is linear function and we look for the coefficients of that linear function. We introduce non-linearities that the data might have into our linear model by adding feature crosses.
For example, lets say we have two-dimensional features ($x_1, x_2$) and label them as $1$ if they are in the 2nd or 4th quadrant, 0 otherwise. A linear function $y(x_1, x_2) = b +m_1x_1+m_2x_2$ cannot approximate this. If we add $x_3 = x_1x_2$, we can find a model that represents that.
However, this is obviously a very artificial problem and not at representative to solve against any kind of data. In the end, we are trying to find a function $y$ that fits some data. This reminds me of solving differential equations numerically. And there we build the function $y$ as a sum of local basis functions $y(x) = \sum_i m_i\phi_i(x)$.
Why would we use global basis functions instead of local basis functions to introduce non-linearity into our linear machine learning model? Why don't we discretize our feature space and optimize on a linear function space?