0
$\begingroup$

I am looking at this tutorial: https://www.dataquest.io/mission/74/getting-started-with-kaggle

Following is the code for linear regression to predict, based on some variables, the survival of the titanic passengers.

# Import the linear regression class
from sklearn.linear_model import LinearRegression
# Sklearn also has a helper that makes it easy to do cross validation
from sklearn.cross_validation import KFold

# The columns we'll use to predict the target
predictors = ["Pclass", "Sex", "Age", "SibSp", "Parch", "Fare", "Embarked"]

# Initialize our algorithm class
alg = LinearRegression()
# Generate cross validation folds for the titanic dataset.  It return the row indices corresponding to train and test.
# We set random_state to ensure we get the same splits every time we run this.
kf = KFold(titanic.shape[0], n_folds=3, random_state=1)

predictions = []
for train, test in kf:
    # The predictors we're using the train the algorithm.  Note how we only take the rows in the train folds.
    train_predictors = (titanic[predictors].iloc[train])
    # The target we're using to train the algorithm.
    train_target = titanic["Survived"].iloc[train]
    # Training the algorithm using the predictors and target.
    alg.fit(train_predictors, train_target)
    # We can now make predictions on the test fold
    test_predictions = alg.predict(titanic[predictors].iloc[test])
    predictions.append(test_predictions)

import numpy as np

# The predictions are in three separate numpy arrays.  Concatenate them into one.  
# We concatenate them on axis 0, as they only have one axis.
predictions = np.concatenate(predictions, axis=0)

# Map predictions to outcomes (only possible outcomes are 1 and 0)
predictions[predictions > .5] = 1
predictions[predictions <=.5] = 0

accuracy = sum(titanic["Survived"] == predictions)/len(predictions)

print (accuracy)

This is clear. For logistic regression is much easier:

from sklearn import cross_validation

# Initialize our algorithm
alg = LogisticRegression()
# Compute the accuracy score for all the cross validation folds.  (much simpler than what we did before!)
scores = cross_validation.cross_val_score(alg, titanic[predictors], titanic["Survived"], cv=3)
# Take the mean of the scores (because we have one for each fold)
print(scores.mean())

Now, why can't I do just, for LinearRegression:

from sklearn import cross_validation

# Initialize our algorithm
alg = LinearRegression()
# Compute the accuracy score for all the cross validation folds.  (much simpler than what we did before!)
scores = cross_validation.cross_val_score(alg, titanic[predictors], titanic["Survived"], cv=3)
# Take the mean of the scores (because we have one for each fold)
print(scores.mean())

This gives me a totally wrong result. Why is that? What is wrong in the last snippet of code?

$\endgroup$
1
$\begingroup$

I suspect that you are missing this mapping:

# Map predictions to outcomes (only possible outcomes are 1 and 0)
predictions[predictions > .5] = 1
predictions[predictions <=.5] = 0

Basically, what linear regression produces after a call to predict is a set of numbers (scores), then we need to manually map it back to class label. cross_validation.cross_val_score doesn't do the mapping for us, so what it ends up doing is comparing scores with class labels, which turns out to be bad.

On the other hand, predict of logistic regression gives us the class label (Reference)

| improve this answer | |
$\endgroup$
  • $\begingroup$ I thought of that, but since I am comparing this to a set of 1 and 0, my score should be 0 (since most results won't be 0 or 1), but I get 0.3 instead of 0.8, and 0.3 is still much higher than 0 $\endgroup$ – user Mar 17 '16 at 21:04
  • $\begingroup$ I re-run your script and I only got 0.13 as mean accuracy. $\endgroup$ – Tu N. Mar 17 '16 at 21:28
  • $\begingroup$ From the page I linked I get: 0.787878787879 [ 0.78451178 0.78787879 0.79124579] $\endgroup$ – user Mar 17 '16 at 21:36
  • $\begingroup$ and 0.374682056691 [ 0.33211124 0.39263029 0.39930464] for linear regression $\endgroup$ – user Mar 17 '16 at 21:36
  • $\begingroup$ how do you deal with missing value? $\endgroup$ – Tu N. Mar 17 '16 at 21:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.