Are linear regression models with non linear basis functions used in practice?

Question

I know that popular linear regression models such as Lasso or Logistic Regression are widely used in practice because they perform reasonably good, are efficient and interpretable.

As far as I know, the only way for these models to learn non-linear relationships between $X$ and $y$ is by applying non-linear transformations to $X$ through basis functions $\phi(X)$ (as it is summarized in the scikit-learn documentation).

I wonder if this approach is actually used in real world problems, where usually we don't have any prior information about how to design $\phi$.

Answer 1

Yep, that's a thing. It's called a "Generalized additive model (GAM)":

https://en.wikipedia.org/wiki/Generalized_additive_model

You may also be interested in "multivariate adaptive regression splines (MARS)":

https://en.wikipedia.org/wiki/Multivariate_adaptive_regression_splines

EDIT: Regarding demonstrating that these are actually used, I'm not sure what you're looking for. I've never seen a survey to try and gauge the popularity of specific models, and I'm not sure how useful such a thing would be. I could speak from my subjective experience, but I don't consider that particularly meaningful either.

If you just want examples:

• Hits for GAM models on PubMed - 2114 hits
• Hits for MARS models on Pubmed - 190 hits

For context:

• Hits for Random Forest models on PubMed - 5604
• Hits for Deep Learning on PubMed - 4435 hits
• Hits for Neural Network on PubMed - 57170 hits

Take these counts with a solid dose of salt: in my experience random forests are in much wider use in practice than neural networks. But like I said earlier, you shouldn't really trust my subjective experience either since that's just anecdotal.

Answer 2

In responses to the comment requesting real world application. I have not found models like GAM used particularly often in the industry, but there were notable exceptions in my career.

Insurance companies, especially life insurance companies, tend to prefer an explainable model over one that just makes good predictions. In this space, I have used GAM to model spatial distribution. I can't go into particular details, but to give you a related example, imagine an experiment design regarding the survey of fish. A properly designed survey will stratified an environment; however, within the environment there exist covariates. The traditional approach would be at apply a GLM and add covariates into the model. However, it's easy to see that some predictors may not be linear. For example, fish are drawn to different temperature, so the relation for the environment of the water and the temperature of the wish should probably not be modeled in a linear fashion. Thankfully we could use a cubic spline instead and produce a readily interpretable result.

Another example, would be in credit scoring. In practice, there's usually a myriad of mixed models working together to produce a credit score and there exist different types of models for different level of products and risk. Generically speaking though, some of these models can be hard to understand and find what driving factors effect it. MARS can be used as a good baseline piecewise model for credit scores.

In short, it's like any other model. There's a use, time and place for it. I've found that companies that rely heavily on more traditional statistics and process understanding are more inclined to use models like GLM or GAM, while newer tech heavy companies favor the more blackbox quick iteration push to production approach. Neither is better than the other, just that each industry has its own standards and goals.