0
$\begingroup$

I just finished working on my first machine learning algorithm i.e Linear regression. I want to reduce the rmse by optimising the model. I found out that gradient decent does the same job. But i dont know how to do it in python. I refered to some videos on youtube but every video explains y=MX+c . But i have about 50 variables in my model.

Is there any library for it. Please help me out.

$\endgroup$
0
$\begingroup$

as Stephen pointed out, gradient descent is mainly an optimization algorithm used as part of many (I would say most of, and its variances as adam) machine learning modelling use cases. Nevertheless, it is good to try to understand it first by applying it to a linear regression model as you asked for. You can take a look at an explanation from scratch in this post: https://medium.com/@German_CM/real-use-case-with-regression-supervised-learning-a3cb69a3ec0d You can also use the scikit learn linear model library, in this case both SGDRegressor and LinearRegression (the classic Ordinary Least Squares) can be used.

$\endgroup$
0
$\begingroup$

You want to implement gradient descent on a multivariate linear regression.

You can find an implementation in python here : https://towardsdatascience.com/implementation-of-multi-variate-linear-regression-in-python-using-gradient-descent-optimization-b02f386425b9

$\endgroup$
0
$\begingroup$

There are several options that you could try. AFAIK, there are no library that implements gradient descent and modelling as separate instance. What is available though, are models that use gradient descent as its means of optimization. So basically the gradient descent is already embedded and automatically used when you fit the model. As for your use cases you can try sgd regressor by sklearn. Also I would suggest you to use PCA to reduce the dimensionality.

The closest thing that I could think of that implement automatic gradient descent is pytorch. So pytorch is a deep learning library but it optimizes by automatic differentiation, basically you can perform forward pass and then when you ask for update (or backward propagation in neural network) and the gradient will be automatically calculated based on operations that you perform during forward pass. You can think of linear regression as a single layer neural network so it will work but probably overkill since there are libraries that implemented simple linear regression better. But since you are new I would not suggest you to use it, but it is worth to check if you are curious.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.