2
$\begingroup$

I'm working on a project in which I'm developing a precipitation forecasting model.

When I try to predict the original data, the model (LSTM neural network) is not able to predict the peaks.

This is the predicted values for the original data: enter image description here So, I decided to perform

1) A differencing method in which I simply subtract values of the past from current values.

2) A moving average method on the resultant data from step 1.

Then I try to predict this processed version of the data.

Below is the predicted values for the smoothed data: enter image description here

1) Is this kind of forecasting models acceptable among academicians?

2) should I try just to predict the original data?

3) What probable usage can these types of predictive models (which forecasts smoothed version of the data) have?

4) Any suggestion on how to deal with this situation?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

In estimation theory, the simplest way to estimate a random variable is to predict the average of its realizations. Very easy, very simple and very naive to be useful.

Let me explain more: in your problem which is the weather, the correlation is very high between the data from the last several days and the next day, which makes your estimation a good one but in many problems, the corelation between the data from previous days and the next day is small ( the stock market). Another thing you need to keep in your mind which is the cost of doing a mistake, meaning, predicting the temperature to be the average of the last few days (75), if the actual temperature is 78, then the difference is 3 may be acceptable, in stocks, if the average from the last few days is 10 and the actual value is 10.5. This difference 0.5 may not be acceptable.

What does mathematically this mean? If the acceptable error in you estimation is << than the variation within the data having an average based estimator is not acceptable.

Regarding the issue of transforming the data from one domain to another, it is okay if you a deterministic mapping function that can take you back to the original domain. We prefer to have one to one mapping

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for the answer. Could you please point out to one of these mapping techniques used in time series analysis? $\endgroup$
    – hyTuev
    Commented Sep 24, 2018 at 18:35
1
$\begingroup$

1,2) In classification and regression problems you are not searching for the average of correct output, but you are trying to predict the correct output! So no, I don't think this is what you want to do. You are considering an easier problem, where the output expected is just the moving average of the real values.

3) I never did forecasting for weather, but I guess in a way having that information is not completely useless. (Nonetheless not correct!)

4) I don't know how much you know about regression, but:

  • have you split the dataset in train, validation and test set?
  • have you considered using CNN for time series
  • How is your accuracy on train e validation? Are you overfitting?
Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks. 1. Actually these predictions are for test validation data. 2. Using CNN in time series seems interesting. 3. It's close to the training data but a bit lower. So, a little overfitting exists. $\endgroup$
    – hyTuev
    Commented Sep 24, 2018 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.