# Regression methods [closed]

I want to understand what regression methods exist and their purpose. I know the least squares method with which you can build a linear and non-linear model and make predictions.

The ARMA model is used to predict stationary time series, and the ARIMA model is used for non-stationary time series. When many data and signs are used neural networks. For time series, LSTMs are popular. What methods are applicable for solving multiple regression problems? What methods should I use in what cases?

• Hi Lionell! This is a very broad question, most probably too broad. If I would like to answer that, I would have to write several pages describing nearly every existing machine learning algorithm. If you're simply looking for an overview, I would suggest, you start by googling the most famous machinen learning algorithms. Nearly all of the famous classification algorithms (SVM, Random Forest, Neural Nets, ...) support regression as well. – georg-un Aug 10 at 8:36
• @georg-un Yes, I am familiar with all of them. I do not understand when to use SVM or Random Forest or Neural Network or even least squares method for prediction. For example, how should I understand what needs to be used SVM? – Lionell Aug 10 at 8:45
• Hi, welcome to the forum. The question is very broad. Maybe rethink your question and post again. – Peter Aug 10 at 9:13

All regression methods have the same purpose, but some methods are better suited to a given problem than others.

For example, algorithms based on decision trees, like random forest or XGBoost, do not extrapolate outside the area that is covered by the input features. On the other hand, they can handle categorical features.

SVMs are not very popular because they are computationally expensive but if you don’t have too much data, then that’s not a problem. Some algorithms, like deep networks, require a lot of data to train, so you can’t use them if you have little data.

Linear regression only models a linear relationship between the input features (x) and the output, but you could add x squared as an input feature to model also quadratic relations. To model complex function shapes though, that quickly becomes unmanageable. If a fairly simple function shape is fine, linear regression generally allows you to understand/explain your model, so that can be a plus.

In this way, you need to understand the pros and cons of each to know in advance which algorithm is going to work well, but how do you find out, eh? It’s like saying, you need experience. So, first, be aware that choosing the best algorithm is normally far less important than making good use of the algorithm you’ve chosen. Second, try it out, get your hands dirty, and you will find what works for what. Finally, start with the simplest option that may work. Eventually, you’ll have “Experience”.