1
$\begingroup$

I am trying to predict flight take off delay using my current dataset. At this point of time, I only have four dimensions.

scheduled_time_departure (A), flight_id, day_of_week actual_time_departure(B), take_off_delay(B-A)

when I try to predict actual_time_departure(B) using scikit linear regression model using (scheduled_time_departure (A), flight_id, day_of_week) on x-axis, I am getting good r2_score However when I am trying to predict take_off_delay (which is actually difference between actual_time_departure and scheduled_time_departure), in that case I am getting negative r2_score.

Note:

  1. to convert string to integer I am using LabelEncoder.
  2. scheduled_time_departure and actual_time_departure are in seconds not a timestamp, that is second of day, 86400 is max value it can have.
  3. I even tried doing normalization of scheduled_time_departure
  4. I have ensured that take_off_delay is always positive.
  5. For the case when I was predicting actual_time_departure, I tried using one hot encoder but that agravated r2_score

P.S: I am new to machine learning and data science, let me know if I am making dumb mistake :) P.P.S: I understand model can be worst if r2 score is negative, however I want to understand the reason.

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ You shouldn't use flight_id in a model, as it unique for each flight and thus doesn't give information (as I understood). Also, you shouldn't use normalization, if there is such simple dependency (difference of two columns). The negative r2 score means that prediction is worse than a mean prediction. Could you try to run training without flight_id and report results? $\endgroup$
    – Viacheslav Komisarenko
    Dec 19, 2018 at 13:11
  • $\begingroup$ @ViacheslavKomisarenko flight_id is something which is unique for given time and route. I have result both with and without normalization, however r2_score was negative in both case when I was predicting take_off_delay. However when I try to predict actual_time_departure I get good r2_score. take_off_delay is absolute delta between actual_time_departure and scheduled_time_departure. Same flight_id can be repeated for multiple days. $\endgroup$
    – Pramod
    Dec 19, 2018 at 16:04
  • $\begingroup$ @ViacheslavKomisarenko flight_id is like bus number, which can have entry for each day. I have also tried this without flight_id but even in those cases r2 score was negative. $\endgroup$
    – Pramod
    Dec 19, 2018 at 16:10
  • $\begingroup$ It is obvious your model is not performing well. And without deep analysis it is hard to give specific guidelines. But I also agree that unique IDs are not useful unless they may appears multiple time (not just few ids being repeated!). Said that I also note that you do not good set of features anyway, so it is not easy to fit a model. Also why linear model? Why don't you try RandomForest or Gradient Boosting Trees? $\endgroup$
    – TwinPenguins
    Dec 19, 2018 at 20:37

1 Answer 1

Reset to default
1
$\begingroup$

A negative $R^2$ on your training set typically means you didn't fit an intercept. A negative $R^2$ on your test set means the model is simply very bad. Indeed flight ID is not a meaningful scalar value and can't be in a linear regression.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I'm not sure that -ve r^2 means the model has no intercept - it means in general it means the model is actually worse than just using the mean of the training data as a prediction - i.e. you've increased rather than reduced the variance of the residuals relative to y - mean(y) $\endgroup$
    – David Waterworth
    Jan 15, 2020 at 3:22
  • $\begingroup$ That's true, but, how does one manage to do that? OP is using linear regression, and if the setup were otherwise fine and normal, that's the only way I think this can arise. Linear regression can always choose 0 coefficients and intercept = mean of the target, so shouldn't do worse. But here, I really think it's the data. The ID is (I'm guessing) huge and meaningless and dominates the regression. $\endgroup$
    – Sean Owen
    Jan 15, 2020 at 5:41
  • 2
    $\begingroup$ Yeah that's true, plus it's not obvious from the original post but it could be the r^2 from a test set rather than the training data. They are learning ML using sklearn so most tutorials will stress doing a test/train split etc. $\endgroup$
    – David Waterworth
    Jan 15, 2020 at 21:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.