4
$\begingroup$

Judging by the negative result being displayed from my ridge.score() I am guessing that I am doing something wrong. Maybe someone could point me in the right direction?

# Create a practice data set for exploring Ridge Regression


data_2 = np.array([[1, 2, 0], [3, 4, 1], [5, 6, 0], [1, 3, 1],
           [3, 5, 1], [1, 7, 0], [1, 8, 1]], dtype=np.float64)


# Separate X and Y

x_2 = data_2[:, [0, 1]]
y_2 = data_2[:, 2]

# Train Test Split
x_2_train, x_2_test, y_2_train, y_2_test = train_test_split(x_2, y_2, random_state=0)

# Scale the training data
scaler_2 = StandardScaler()
scaler_2.fit(x_2_train)
x_2_transformed = scaler_2.transform(x_2_train)

# Ridge Regression
ridge_2 = Ridge().fit(x_2_transformed, y_2_train)
x_2_test_scaled = scaler_2.transform(x_2_test)
ridge_2.score(x_2_test_scaled, y_2_test)

Output is: -4.47

EDIT: From reading the scikit learn docs this value is the R$^2$ value. I guess the question is though, how do we interpret this?

Improve this question
$\endgroup$
2
  • $\begingroup$ That's a very small data set. How many points are in your test set? Also, how did you generate those points? There are some good facilities with sklearn to generate synthetic data sets that might make more sense. $\endgroup$
    – Wes
    Feb 16, 2019 at 0:33
  • 1
    $\begingroup$ FYI this often come up when you don't fit an intercept, but that's not the issue here. $\endgroup$
    – Sean Owen
    Feb 16, 2019 at 0:42

2 Answers 2

Reset to default
3
$\begingroup$

A negative value means you're getting a terrible fit - which makes sense if you create a test set that doesn't have the same distribution as the training set.

From the sklearn documentation:

The coefficient $R^2$ is defined as (1 - u/v), where u is the residual sum of squares ((y_true - y_pred) ** 2).sum() and v is the total sum of squares ((y_true - y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a $R^2$ score of 0.0.

Improve this answer
$\endgroup$
4
  • $\begingroup$ This makes more sense. What exactly do you mean by different distributions though? $\endgroup$
    – Ethan
    Feb 16, 2019 at 0:38
  • $\begingroup$ I mean that samples in the training set and test set were created from the same process. $\endgroup$
    – Wes
    Feb 16, 2019 at 0:41
  • 1
    $\begingroup$ I agree, and to expand on that, it's not that you necessarily create the test/train sets from different distributions, but both sets are so small that whatever 1-2 examples it chooses from your 7 is as good as from a very different distribution. Another way to say it is: I'm not sure your made-up test set even looks like it comes from a linear relationship, which you're trying to fit. $\endgroup$
    – Sean Owen
    Feb 16, 2019 at 0:44
  • $\begingroup$ Yes, that's another good point. It might just mean that a linear model is not sufficient to model this data set. $\endgroup$
    – Wes
    Feb 16, 2019 at 0:48
0
$\begingroup$

To understand what negative value of coefficient of determination ($r^2$). You need to know what $r^2$ = 0 means.

$r^2$ = 0 means that the squared error of your regressor fit is same as the squared error for a fit that always returns the mean of your targets.

If $r^2$ is negative it means that your regressor fit has a higher squared error than the mean fit. That is, it performs worse than the mean fit.

$r^2$ = 1 - Squared error(your fit)/Squared error(mean fit)

`

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.