# Tag Info

0

I don't really understand your problem statement. You can simply add squared terms to any linear model. Say you have $y$ and $x$ and you want to model a polynomial function, you can write a model like: $y=\beta_0 + \beta_1 x_1 + \beta_2 x_1^2 + u$. In matrix form this would look like $y=X\beta + u$. An example with some numbers would write: $\left( \... 1 One mistake I see is the fact that you are using the learning rate twice, one when calculating the partial derivatives and once when updating b0 and b1. The learning rate should only be used when updating b0 and b1 and is not used in calculating the partial derivates to b0 and b1. 0 You could combine these features before using one-hot encoding, and see if the performance is improved. But keep in mind, that it really depends on the problem each time. Generally, is a good thought to combine these type of features. CatBoost, a very good gradient boosting library, create such combinations and the results are pretty good most of the time. ... 3 It's not actually possible to directly compare model coefficients. What you might do that would make more sense is to compare similar metrics. A good start would be to learn about explainability metrics that are comparable across models : LIME, SHAP... etc. (see here : https://christophm.github.io/interpretable-ml-book/) to see how models reacts on ... 3 Weights across different types of models are not always comparable, so I think that it would make more sense to do this kind of comparison not across different types of model but within a single type of model varying: the hyper-parameters (if any), the set of instances (e.g. selecting different subsets randomly), the set of features. I'd recommend in ... 0 Looking at https://en.wikipedia.org/wiki/Simple_linear_regression : This t-value has a Student's t-distribution with$n-2$degrees of freedom. Using it we can construct a confidence interval for$\beta$: $$\beta \in \left[\widehat\beta - s_{\widehat\beta} t^*_{n - 2},\ \widehat\beta + s_{\widehat\beta} t^*_{n - 2}\right]$$ at confidence level$1-\gamma$,... 3 However Root Mean Square Error seems similar to MSE and is the root of it, gradient of RMSE with respect to$i^{th}$prediction differes from that of MSE. $$\frac{\sigma{RMSE}}{\sigma{y_i}} =\frac{1}{2}\frac{1}{\sqrt{MSE}}\frac{\sigma MSE}{\sigma y_i}$$ Gradient of RMSE equals to the gradient of MSE multiplied by this$\frac{1}{2}\frac{1}{\sqrt{MSE}}\$ ...

0

Two reasons, mainly the second I think. The RMSE is an indication of the noise levels in the scale of standard deviations. The RMSE has nice mathematical properties for fast calculations. (Its gradient is linear and propagates easily)

0

I have two more things to add: First, try also interaction effects, i.e. when you add variables such as: y = A*x1 + B*x2 + A*B*x3 In this way, a new parameter enters the regression to account for the interdependent effects of A and B. It's a trick to move beyond the linearity assumption, and force the impact of one explanatory variable on y as dependent ...

0

Answer based on the comment: Intention here is to reduce the error by all means and considerations Choice of model: First let's address the obvious assumption: linear regression is a model which requires the response variable to be expressed as a linear combination of the independent variables. In order to improve performance in general one must make ...

Top 50 recent answers are included