Hot answers tagged

3

sklearn has such a functionality already for regression problems, in enet_path and lasso_path. There's an example notebook here. Those functions have some cython base to them, so are probably substantially faster than your version. One other improvement that you can include in your implementation without adding cython is to use "warm starts": ...


2

Yes the L1 regularization will shrink the irrelevant feature coefficients to zero and hence it doesn't require feature selection. In fact it IS a commonly used feature selection technique. So basically you are performing feature selection!!


2

Interesting question. I'd say it is correct not to divide, due to the following reasoning... For linear regression there is no difference. The optimum of the cost function stays the same, regardless how it is scaled. When doing Ridge or Lasso, the division affects the relative importance between the least-squares and the regularization parts of the cost ...


2

When we implement penalized regression models we are saying that we are going to add a penalty to the sum of the squared errors. Recall that the sum of squared errors is the following and that we are trying to minimize this value with Least Squares Regression: $$SSE = \sum_{i=1}^{n}(y_i-\hat{y_i})^2$$ When the model overfits or there is collinearity present, ...


1

These diagrams show the "constrained" version of lasso/ridge, in which you minimize the pure loss function subject to a constraint $\|\beta\|_1\leq t$ or $\|\beta\|_2\leq t$. (Another common version adds a penalty to the loss, and these are equivalent.) The bluish solid shapes are the set of points with $\|\beta\|\leq t$, on the left with L1 norm ...


1

Lasso does feature selection in the way that a penalty is added to the OLS loss function (see figure below). So you can say that features with low "impact" will be "shrunken" by the penalty term (you "regulate" the features). Because of the L1 penalty, the $\beta_i$ can become zero (which is not the case with Ridge, L2). In the ...


1

Kaggle is a crowd source platform with no quality control. It is to be expected that there will be deviations from best practices.


1

In answer to your first question: The reason that your RMSE proceeded to increase as you increased the strength of your regularization (the value of $\lambda$) can be explained by reviewing the intuition behind what is happening when you increase the regularization of your model. Why did could my RMSE have kept increasing as I increased my regularization ...


1

Standard Scalar trained on 30 features so it expects 30 features only. One simple hack you can do is, you can create a new Standard Scalar and train with those 20 features, and replace your pipeline Standard Scalar with the new one. For the LogisticRegression, get the non zero weights and set those weights to the new model with 20 features without any ...


1

Change (search over) the penalty parameter of lasso. FinalRevenue = RevenueSoFar is a good baseline "model," but hopefully your other features can improve on that. You might also consider just modeling the target FinalRevenue - RevenueSoFar.


1

Neural Networks are notoriously good at performance and bad at interpretability, i.e. it's very difficult (almost impossible) to explain why a particular prediction was made. It's even more difficult to link the prediction with the features, since the NN does a lot of intermediate calculations where all the features play a role. The notoriously good models ...


1

@Ethan is correct about the formulation of the lasso penalty, and I think it's particularly important to understand it in that form (for one thing, because that same penalty can work with other models like neural networks, tree models, generalized linear models, ...). But, to your question: If $\lambda=0.5$ then does it mean that those coefficients whose ...


Only top voted, non community-wiki answers of a minimum length are eligible