Forecasting with a Machine Learning Algorithm

Question

Im sorry if it is a too general question, but i am stuck somewhere between perfect and adequate in my model. So, i wanted to ask here. If it is not a suitable question, your negative feedbacks are all welcome, sorry. But theoretically subject is inside a gray area and all comments are valuable i think. It is not a widespread approach to use ML for forecasting, as far as I know.

Our sales team publishes a forecasting report at the beginning of each year. But it is mostly far from reality, not very scientific. It includes subjective assesments also.

We decided to construct a ML model to make a forecasting. Not a classical forecasting because there are many input variables that can effect the output. So, it is an experimental project actually.

So far, i have developed an adequate model, which is already far more better than sales one. But still i think it is not a good one, only adequate.

Data includes Customer-FromCountry-ToCountry based sales. So there are many combinations. By looking at similar applications at MachineLearningMastery.com, i have developed my model like that:

customer-from-to-M9-M8.....M1 - month - quarter - movingavg3 - mov6 - mov9 - dptcountryholidaycount - arvcountryholidaycount

m9 to m1 are previous 9 month sales of predicted month M(output variable) M's vary from 0 to 800 mostly. Holidays are from 0 to 6 for example. I Scaled all of these numerical variables between 0-1

Customer-From-To-Month-Quarter variables are categorical and i used OneHotEncoding for these.

I shuffled all the data, so there is not a time ordered dependency inside my dataset.

I have divided the data into 3 segments. 1: above 100 average for last 3 months; 2:10-100 average 3: below 0 average for 3 months. I run different models for each 3 segments.

There are many 0 values, so i am using a LGBClassifier first to determine if the sales is 0 or not. Then if it is not 0, i am using a LGBRegressor to determine the sales value for that input row. (SGDRegressor for segment 1, it performs better)

I tried to add MonthSeasonality index and QuarterSeasinality index for each combination into dataset as variables. But it didn't perform well.

What can i try to get better score, im not asking about algorithms etc. My main purpose here is to get insights about construction of dataset, preprocessing. Your precious ideas about preprocessing tricks before applying a ML algorithm.

Thanks

Answer 1

I'm afraid your data is probably too complex and specific for somebody else to understand exactly what's going on.

The only idea I can suggest is to try to analyze manually the errors that your model makes:

• Are there any patterns, like a kind of errors which happens quite often? For instance a country which tends to be overestimated, a month which tends to be under-estimated, this kind of thing.
• If yes try to investigate why: for instance is there some additional feature that could help the model make better predictions for these cases?