We have a data set of 300,000+ records that looks something like the following:

Item ID, Quantity 2017, Quantity 2016, Quantity 2015, Quantity 2014, Quantity 2013
1111, 100, 50, 25, 10, 0
2222, 0, 10, 100, 500, 1000
3333, 10, 0, 5, 2, 4

We are currently attempting to find the best model to predict Quantity 2017 based on the previous quantities for each record. We have tried Decision Tree Regression, Multiple Linear Regression, and Random Forest Regression (10, 100, 1000 trees) but our results were a bit too far off for approval. We are using 80% of our data for training and 20% for testing.

Are there any models that are better suited for this type of calculation?

We are also concerned that perhaps our data structure is part of the problem and we should re-evaluate. Is there another structure that might be better?


It looks to me that the data are in fact a Time Serie. Therefore, you can treat them as such.

Now, there are many approaches with each having each own strengths/weaknesses. I'd start with simple models such as ARMA or SARIMA. Possibly also GARCH or VAR. If you wish to go for something more advanced, Recurrent Neural Networks are also used in Time Series prediction.

Beside that, I am surprised that for example Random Forest have not given sufficient performance. Have you tried thorough feature selection? It may be that you have too high expectations.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.