4

Yes, ridge regression is ordinary least squares regression with an L2 penalty term on the weights in the loss function. The loss function is not really linear in any of its terms, right? it's the squared residuals plus squares of weights. But there is no reason the loss function needs to be linear. It's helpful if it's convex, which it is here, but even that ...


3

370 rows, with approx. 900 features is not optimal. I would suggest some dimension reduction. PCA, factor analysis, PLS regression are some alternatives. You could try a lasso- / elasticnet -regression as well. Here is a good guide. https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html


3

ISL (page261) gives some instructive details. The linear regression loss function is simply augmented by a penalty term in an additive way.


3

Understanding the difference with an example will be very easy. Classification:- When you are asked to predict whether a patient will survive or no from a disease X given all the necessary data of the patients who survived or died due to the same disease X in the past and also given data for predicting the same on the current dataset. Regression:- When you ...


2

As I know we have below equation for Ridge Regression: \begin{equation} RSS_{Ridge} = \Sigma_{i=1}^{n} (\hat{y}_{i} - y_{i})^2 - \lambda \Sigma_{j=1}^{p}(\beta^2) \end{equation} First of all, it seems to me, if lambda goes higher does not mean that coefficients go down with inverse relation to lambda. Because the power of beta is two and the lambda is one. ...


2

Yes and no The YES Let's consider at two distributions $F_X(x)$ and $G_Y(y)$ whose joint distribution is simply the product of the the two distributions: $H_{X,Y}(x,y) = F_X(x)G_Y(y)$. This means that the distributions are 100% independent: not just linearly (no correlation), but that there is absolutely no dependence between the two distributions. (Copula,...


2

Beeing X in the future and beeing X in specific time in the future is just a subset of the first one. So what one really needs to do is just determine the probabilities (or parameters that give us these probabilities) P(X|t>30) Where you can model t, also as your feature. So just fit a model on this data, where you have mutliclassification of: dead ...


1

Looks like your data are severely unbalanced if they are almost all zeros. Try data balancing or some form of cross-validation. To your original point: there is not a simple R^2 metric fix.


1

So, the direct answer here is clearly NO. The answer comes from the definitions of classification and regression. In a classification task what a model predicts is the probability of an instance to belong to a class (e.g. 'image with clouds' vs 'image without clouds' ), in regression you are trying to predict continuous values (e.g. the level of '...


1

R2 of 0.83 means that approx 83% of the variance can be explained by the model, 17% of the variance can't be explained by the model. This is one of the measures to evaluate a model, but you shouldn't try and maximize any number, just for the sake of maximizing the number. An high R2 could be a sign of over fitting, and a model with low R2 could still be ...


1

Shap values is your friend. model = RandomForestRegressor(max_depth=6, random_state=0, n_estimators=10) model.fit(X_train, Y_train) import shap shap_values = shap.TreeExplainer(model).shap_values(X_train) shap.summary_plot(shap_values, X_train, plot_type="bar") There are several other techniques with their own drawbacks. Some are: Permutation Importance ...


1

Your $R^2$ scores indicate that a linear model does not describe your data well. On top of this, there seems to be a large variability in data. You could try the following: If the linear model is supposed to describe the data, check for outliers. They might be responsible for the large variation across the CV folds. Try reducing the number of features if ...


1

Interesting question. As you know, indeed, it can be seen as a pure optimization problem. For that you need to come up with mathematical modeling of dependent variable based on independent ones. I might have an easier ML-based suggestion: Create a new target out of all 3 variabales in a way you want them to behave. For instance, you said bid and cost low ...


1

Good job looking at the tree and understanding what has happened. There is no problem splitting on the same feature multiple times. A continuous feature has many split points available. The tree continues to subset and refine. The split criteria shows what will be the "best" greedy split at this point. If a feature is income, perhaps the best split is \$100,...


1

These are not (direct) assumptions for linear regression. But rather for OLS-Ordinary Least Squares which is widely used to estimate the parameter of a linear regression model. OLS estimators minimize the sum of the squared errors (a difference between observed values and predicted values). In order to guarantee finding best possible parameters, we make ...


1

To answer: 1) When calculating the loss function for multivariate outputs, keras calculates it as a mean across all your outputs: https://github.com/keras-team/keras/blob/2.0.4/keras/losses.py#L12 Hence, if one output is doing really badly and others not, it could influence your loss result. 2) In the source code there are no mentioning about scaling the ...


1

They all start from the same assumption: time series forecasting can't be treated as a regression/classification problem. It is time dependent, which means our target y at time t depends on what the value y was at t-1. Time series forecasting must take into account time dependency, but it doesn't have to be the only source of information. Many complex ...


1

One option would be Huber Loss which could be setup to increase the weight for some types of errors compared to other errors.


Only top voted, non community-wiki answers of a minimum length are eligible