Tag Info

Hot answers tagged regression

6

GridSearchCV is built around cross validation, but if speed is your main concern, you may be able to get better performance using a smaller number of folds. From the docs: class sklearn.model_selection.GridSearchCV(estimator, param_grid, scoring=None, n_jobs=None, iid='deprecated', refit=True, cv=None, verbose=0, pre_dispatch='2*n_jobs', error_score=nan, ...

5

Problem in this competition is VERY similiar. Instead of me copy-pasting some interesting ideas just check out winner write-ups in the discussions. Some takeways: Add count of values of features as new features. This is information that LightGBM can’t see Check unique value counts for all features of train and test set. Model stacking almost always works ...

5

If you want to move from theory to application then I suggest to do exactly that: get your handy "dirty"! UCI Machine Learning Repository has some easier datasets to get started. Kaggle is great too but before going for any competition look for an easier dataset from their repository. If you prefer something with more guidance the book "Introduction to ...

4

Doing Kaggle problems is a good way to test your skills, and it is a good way to improve your skills. There are problems that don't require advanced techniques. For example, Titanic is an introductory problem. Also, solutions for many problems are available. You can do a problem yourself and then check how other people did it.

2

If you up to a predictive model, you look for a model which performs well on the test set and the metric of interest is the mean squared error which indicates by how much you fail to predict $y$ on average. So don't use $R^2$. Just compare all models based on MSE.

2

Question 1. Both. If you think in opposite to multivariate terms, than in univariate regression both input and output variables should be 1-d Question 2. Multivariate regression where more than one independent variable (predictors) and more than one dependent variable (responses), are linearly related. So input needs to be more than 2 also.

2

A few hints/ideas regarding your task: I would not kick out other explanatory variables $x$ too early. When I look at your correlation chart, I suspect that all variables (but serial no) have some impact on $y$. If you kick out these variables, you may throw away important information. In case some $x$ are highly correlated, you may face the problem of ...

2

By passing a callable for parameter scoring, that uses the model's oob score directly and completely ignores the passed data, you should be able to make the GridSearchCV act the way you want it to. Just pass a single split for the cv parameter, as @jncranton suggests; you can even go further and make that single split use all the data for the training ...

2

Weighting MSE is a way to give more importance to some prediction errors than to others in the overall score. This is useful if you are using MSE as a performance metric for your model, especially during the model training (loss function) or validation (hyper-parameter setting). In the case you cite as example, more importance is given to cases with more ...

1

Is 0.9113458623386644 my ridge regression accuracy(R squred) ? if it is, then what is meaning of 0.909695864130532 value. These are both R^2 values: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html#sklearn.linear_model.Ridge.score The first score is the cross-validation score on the training set, and the second is your test ...

1

If speed is the only only issue then i have few suggestions that will definitely improve the algorithm run time by 5-10times(which i experienced), without compromising on any other input: 1) Increase the number of jobs submitted in parallel, use (n_jobs = -1) in the algorithm parameters. This will run the algo in parallel instead of series(and will cut down ...

1

There are a few ways of making this faster: Decrease the CV value, as mentioned by @jncraton Decrease the search space for the hyperparameters (test only a few parameters or decrease the ranges for parameters) Additionally, you might consider using a more efficient way of hyperparameter searching by using hyperopt or nevergrad.

1

I agree with @peter's detailed answer on points #1 through #5 and would like to supplement that with some more details: Perform a PCA or MFA of the correlated variables and check how many predictors from this step explain all the correlation. For example, highly correlated variables might cause the first component of PCA to explain 95% of the variances in ...

1

I'll go through your question one by one. 1) Can someone explain why we have to transform dependent variable using log-transformation (Normalization) when appear positive skewed y variable in regression model? Not necessarily log transformations, any kind of transformation (square, square-root, log, Z-scores, you name it) necessary to make the ...

1

Just a couple thoughts: It looks like these "regimes" could be represented as a latent variable: you could probably design a bayesian model in which the OLS model depends on the value of this latent variable. This means that the model would still be trained only with the observed features, but would internally predict the value of the regime and this value ...

1

A good place to start is with Analysis of Variance (ANOVA) models. The simplest case is where the response/outcome variable is continuous and you have 1 categorical predictor. This is called one-way ANOVA. With 2 categorical predictors you have a 2-way ANOVA and so on. With more than one predictor, interactions between the predictors are also typically ...

1

Your question boils down to what the difference between $R^2$ and $\bar{R^2}$ is. R-squared is given by: $$R^2=1-(SSR/n)/(SST/n) .$$ The adjusted R-squared is given by: $$\bar{R^2}=1- [ SSR/(n-k-1)]/[SST/(n-1) ].$$ $SSR$ is the sum of squared residuals $\sum u_i^2$, $SST$ is the total sum of squares $(y-\bar{y})^2$, $n$ is the number of observations, ...

1

Lambda is a tuning parameter („how much regularisation“, I think called alpha in sklearn) and you would choose lambda so that you optimise fit (e.g. by MSE). You can do this by running cross validation. This page (for the GLMnet package in R) explains how to apply Lasso in a very instructive way (alpha is the elastic-net mixing parameter here, Lambda is ...

1

I just want to stress an important point: ConvLSTM() layers have been excluded from the new TensorFlow 2.0, which is largely based on Keras in models' specification. It is substituted by ConvLSTM2D() layers, that take different arguments as input. (see docs here). (An alternative is to manually create a combination of Conv2D() and LSTM() layers.) That is to ...

Only top voted, non community-wiki answers of a minimum length are eligible