0
$\begingroup$

I am given an Excel pivot table that aggregates data from a somewhat sizable data source (a database table with 1.9m records and another of about 490k). The data within the Excel file consists of 3 columns: dates of Mondays which represent their respective weeks, quantity of items, and number of shipments (that it takes for the quantity of items). I am supposed to concoct a model that predicts the number of shipments that would be required for a given quantity of items in the future. What models could I implement for such a small dataset with just 1 explanatory and 1 response variable? I know the run-of-the-mill linear regression with a confidence interval would be a start but the data has a dense cluster and then sparse data with some positive correlation. The color bar represents the date (purple is earlier, yellow is most recent)

enter image description here

$\endgroup$
  • $\begingroup$ An important note, because of the nature of your task and the sparse data a baseline "score" against which to measure your model is critical. Assume your business has to solve this issue anyway. Without any ML insights you might assume the "average" shipment count or something equally simple. What is the evaluation of that (i.e. MAPE, MSE, etc.) and how does it fare against the simplest linear regressions, etc. Knowing this will help determining whether you add value or not and what effort is warranted. $\endgroup$ – Fnguyen Apr 23 at 13:26
0
$\begingroup$

Create additional features from the given features. From the Monday "date" feature, the features ("month","year") can be found. Additionally, months of the year contain either 4 or 5 Mondays, so a "week" feature can be created with a value 1...5 that can be used to represent the week of the month that the Monday occurred on. Additional features can be created by calculating trends (Average Shipments for the previous N shipments,...etc.). This gives the algorithm more information to work with and should provide more refined results.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.