Timeline for regression model outperform every models
Current License: CC BY-SA 4.0
20 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 21 at 19:12 | vote | accept | S. M. | ||
| Nov 20 at 18:09 | answer | added | Robert Long | timeline score: 4 | |
| Nov 20 at 13:04 | comment | added | Robert Long | OK, will do so this afternoon :) | |
| Nov 20 at 12:04 | comment | added | S. M. | @RobertLong would you kindly write review of code? Although I am working on your suggestions. | |
| Nov 19 at 10:19 | comment | added | Robert Long | I share @Mario 's concern regarding data splitting (could introduce data leakage, potentially explaining the unexpected performance), and also their concern about model evaluation. Scaling the entire dataset before splitting risks leaking test set information. Ensure the scaler is fit on the training data only,then applied to both training and test sets. Also, feature-target preparation doesn’t clearly separate training and test sets, which could lead to overlap. For time series (ts) data, a temporal split or rolling-window approach is important. Finally, use proper ts cross-validation | |
| Nov 19 at 6:07 | comment | added | Mario | "heatmap for which matrices? Actual or predicted?" You can use the heatmap to visualize any matrix in the context of predicted or actual or their differences according to $MSE$ or $RMSE$ The same problem has been indicated in Figure 16 in this paper | |
| Nov 19 at 5:54 | comment | added | Mario |
I saw your post there on cross-validated community and @Dave's answer. For better evaluation besides Residuals (MSE), be sure to do something like this then plot the heatmap of the matrix. I think your implementation to feed data for this type of matrix-based multi-variante time is not correct. you need to re-visit it. you might need to use MultiOutputRegressor. I liked you started to ask questions from scratch but the implementation looks for me invalid
|
|
| Nov 19 at 5:53 | comment | added | S. M. | @Mario heatmap for which matrices? Actual or predicted? | |
| Nov 19 at 5:43 | comment | added | S. M. | @Mario would you provide code snippet of any models, I have implemented the code based on this answer that I asked in cross-validated site. | |
| Nov 19 at 5:38 | comment | added | Mario |
To the best of my knowledge, Seeing this, and you need to apply a Multi-variant time-series (matrix of 24x 25). then y ${y_0, ... ,y_{24}}$ but this needs to be fetched to models in batches of 24 so that the model generalizes it. wherever you select as a time-friendly model at the end plot the difference in predicted matrix = Predicted Matrix - actual Matrix using $MSE$ formula for each pixel. showing the Residuals (MSE) in this form does not indicate whether model X outperform or not unless you do average on all predicted matrices
|
|
| Nov 19 at 5:34 | comment | added | S. M. | @Mario would you found any bug in my code(with respect to my current scenario which is train by three consecutive days for each models)? | |
| Nov 19 at 5:26 | comment | added | S. M. | @Mario no, it's my requirement to use consecutive three days for training. | |
| Nov 19 at 5:20 | comment | added | Mario | I already read your previous question. To be realistic, let's plot the first month (first 31 consecutive days), and also you need to try to visualize a bit of results predicted matrixes to see if the used models could capture relations. I'm not sure if I see the problem Multi-variant time-series (matrix of 24x 25) problem, the time-horizon you considered for case1 is good enough! see this, maybe you consider 7 consecutive days for train | |
| Nov 19 at 5:16 | comment | added | S. M. | @Mario arima model take time to run, every day around take 45 seconds. | |
| Nov 19 at 5:14 | history | edited | S. M. | CC BY-SA 4.0 |
added 98 characters in body
|
| Nov 19 at 5:04 | comment | added | S. M. | @Mario please have a look on linked question, I have explained there very well. | |
| Nov 19 at 4:55 | comment | added | Mario |
First, let's understand a bit of the data in the form of a matrix of 24x25 called pixels before applying regressions on 25 columns. I plotted the last month or 31 days. Based on your Case1: Train for consecutive 3 days to predict each fourth day. but seems it would not work very well based on your case1 scenario even though you apply. I'm not sure the output plot you offered Residuals (MSE) is something you claimed to show. I am assuming you take 1st 3-day info out of 93 files and start prediction and you come up with the remaining 90-day results and show Residuals (MSE)
|
|
| S Nov 19 at 4:02 | history | suggested | Mario | CC BY-SA 4.0 |
Edit and grammar correction in title and question body for better understanding
|
| Nov 19 at 3:56 | review | Suggested edits | |||
| S Nov 19 at 4:02 | |||||
| Nov 19 at 3:30 | history | asked | S. M. | CC BY-SA 4.0 |
