Low scale ML/statistical techniques for data poor settings

Question

I have two separate problems, but both suffer from a paucity of data problems:

1. logistic regression
2. Time series prediction

• For logistic regression, I have a tiny dataset with 10 observations which have variables such as:

  age, Marital_Status, income, gender, and car_purchase_status (outcome flag with Yes/No values).

  Now I have a new 11th customer with variables such as Age, gender, Marital_Status, and income. Now I would like to know whether this 11th customer will buy a car or not.

  Should I spend resources to influence him to buy a car? Am I spending my resources on the right customer? For example: Is there any way that I can find out that the 11th customer has 70% or 80% pc chance of buying a car?

  So spending some marketing efforts such as calls can help us convince him to buy a car (100%). So, how can I do this? Any advice, please? Should I just give up straight away that this problem is impossible to solve with such low data, or are there any simple statistical techniques that can help me gain some insights about the 11th customer?

• For Time series prediction, I have only 10 observations, each spaced at 20 days gap. For ex: I have their revenue generated for day1,day21,day41,day61,day81.....day201.

  Now, with the given 10 observations, I would like to predict the revenue generated for day500, day321, day621 etc.

  So, is it possible to run time series forecasting with such a short series? Can you guys guide me on this, please? Here, also, should I give up because of low data points, or are there any methods that I can use to predict the future timestamp points based on short input time series?

Can you guys please help me with the list of steps/topics that can help me do this?

Answer 1

Generally, I'd pick a very simple, transparent/explainable model and use the results in a semi-automated way. That is, do not just derive a prediction but rather insights. You could, for example, use a (or multiple) decision tree(s) which you pre or post prune. The result could be a tree with, let's say, just 1-3 features to find simple rules like "if a customer is married and at least X years old, they have a high chance of making a purchase". With logistic regression you may use coefficients to identify features which influence the dependent variable the most. These (qualitative or semi-quantitative) rules should then be validated with domain experts.

Moreover, you need to be transparent about the accuracy and precision of your estimates. In the above example, leave purity would provide some intuition. If you report any quantitative measures (which I'd be careful with), you may want to consider confidence intervals (see here or chapter 5 in Tom Mitchell's "Machine Learning", for example). (With only 10 samples typical assumptions about normal distribution will not hold here though)

Regarding the time series I would start even simpler. Depending on the number of customers, I'd start by visualizing some or all historical data in a line plot (sales per customer over time) and check the min, max and mean per customer. This gives some intuition regarding potential trends. For example, if all observations remain constant over time for a given customer, whether there is an upward/downward trend or if the data has high variance with no clear trend. Also, there may be clusters of customers which show similar patterns. Obviously, this is neither Machine Learning nor a rigorous statistical analysis but rather a pragmatic approach supported by some basic data analytics.

What you need to be very careful with is the time horizon of any quantitative prediction: based on 10 observations at $t \in \{1,21,...,201\}$ you may derive some conclusions for, let's say, $t<=301$ (to make up a total out of the blue ballpark figure) but $t=621$ is very far in the future. Also, you need to keep seasonality in mind. For example: If your observations are all from October to April of a given year and assuming you have a winter/summer seasonal pattern. Then you cannot infer a lot for months May to September. To understand the limitations and forecast potential of your time series better, I'd speak to subject matter experts, e.g. in sales & marketing. It could also be helpful to understand their forecasting approach and cross check any insights you derive with their predictions.

But, as Erwan pointed out, be very careful to derive conclusions. Applying some "ML magic" will not find a useful pattern if there insufficient data to find any signal. And of course additional data collection would be reasonable if that is an option.

Answer 2

The main problem with very little data is that it's almost impossible to know how representative the sample is. Some people would even say that less 20-30 data points cannot be representative of anything. Every single data point can have a huge impact on any model, so any prediction has a huge margin of error.

If one is going to train a model from a tiny sample, they should do everything possible to avoid overfitting: use very few features, use a very simple type of model with as few parameters as possible. Anything else is guaranteed to overfit.

In your example the behaviour of a customer can be very complex in reality, so it's particularly challenging (and potentially risky) to rely on such a small sample. Imho the first task is borderline but it can be tried, but I don't think the second one is realistic: revenue is not always regular, predictions in the long term are going to be barely better than random.