Timeline for Regression - Model looks good overall but R2 unexpectedly negatively high
Current License: CC BY-SA 4.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 3 at 21:15 | comment | converted from answer | Richard Careaga | See this recent answer on the application of OLS to time series data, which it appears to be what you are working with. What you are seeing may be an artifact of autocorrelation. | |
| Dec 2 at 20:16 | comment | added | Dave | I find graphs like yours to be problematic. At least my eyes look at Euclidean distance between the curves. However, what we care about is vertical distance. If you graph a scatterplot of the true and predicted values, how does it look? You seem to have a number of bad misses. $//$ Depending on the particular calculation, $R^2<0$ is totally reasonable, despite being a "squared" quantity. Sure, $R^2<0$ is undesirable, but it makes sense and is informative in that is flags the predictions as being worse than that of a benchmark model that always predicts $\bar y$. | |
| Dec 2 at 20:06 | comment | added | Debadri Dutta | my point is if the R2 is so highly negative then the graph, the MAPE and the MAE really doesn't make sense. I am unable to understand what's the issue in this case | |
| Dec 2 at 20:05 | comment | added | Debadri Dutta | 2% error rate on train, 4% on validation, considering my range of target variable, it is 800-1200, so, 25-45 is a good MAE I would say as well as the graph | |
| Dec 2 at 20:03 | comment | added | Dave | But how do you assess those to be good values? | |
| Dec 2 at 20:02 | comment | added | Debadri Dutta | I am getting a MAPE of 0.02 on training data and 0.04 on validation data. MAE is 25 on train and 45 on validation. Also, going by the definition of R2, don't you think the graph is not justified for such a high negative R2 | |
| Dec 2 at 18:44 | comment | added | Dave |
The MAE and MAPE are really decent enough How do you assess that to be true?
|
|
| Dec 2 at 18:42 | comment | added | Debadri Dutta | I have around 160 rows and 15-20 feature columns shortlisted from 40-45 columns using RFECV | |
| Dec 2 at 12:48 | comment | added | Robert Long | How many variables do you have ? As a starting point, could you show us a simple linear model with the same variables/features as the model that produced that plot, using ‘lm’ if using R, or ‘OLS’ if using Python/statsmodels. Just edit the question and add the summary output of the model. | |
| Dec 2 at 4:51 | history | asked | Debadri Dutta | CC BY-SA 4.0 |