2
$\begingroup$

I have two separate problems. One is logistic regression and other is time series prediction. But both suffer from paucity of data problems

a) For logistic regression, I have tiny dataset with 10 observations which has variables such as age, Marital_Status, income, gender and car_purchase_status (outcome flag with Yes/No values). Now I have a new 11th customer with variables such as Age, gender, Marital_Status and income. Now I would like to know whether this 11th customer will buy a car or not. Should I spend resources to influence him to buy a car? Am I spending my resources on the right customer? For ex: Is there anyway that I can find out that 11th customer has 70% or 80% pc chance of buying a car. So spending some marketing efforts such as calls can help us convince him to buy a car (100%). So, how can I do this? Any advice please? Should I just give up straight away that this problem impossible to solve with such low data or is there any simple statistical techniques that can help me gain some insights about 11th customer?

b) For time series, I have only 10 observations each spaced at 20 days gap. For ex: I have their revenue generated for day1,day21,day41,day61,day81.....day201. Now with the given 10 observations, I would like to predict the revenue generated for day500, day321, day621 etc. So, is it possible to run time series forecasting with such short series? Can you guys guide me on this please? Here, also should I give up because of low data points or is there any methods that I can use to predict the future timestamp points based on short input time series?

Can you guys please help me with list of steps/topics that can help me do this?

$\endgroup$
3
$\begingroup$

Generally, I'd pick a very simple, transparent/explainable model and use the results in a semi-automated way. That is, do not just derive a prediction but rather insights. You could, for example, use a (or multiple) decision tree(s) which you pre or post prune. The result could be a tree with, let's say, just 1-3 features to find simple rules like "if a customer is married and at least X years old, they have a high chance of making a purchase". With logistic regression you may use coefficients to identify features which influence the dependent variable the most. These (qualitative or semi-quantitative) rules should then be validated with domain experts.

Moreover, you need to be transparent about the accuracy and precision of your estimates. In the above example, leave purity would provide some intuition. If you report any quantitative measures (which I'd be careful with), you may want to consider confidence intervals (see here or chapter 5 in Tom Mitchell's "Machine Learning", for example). (With only 10 samples typical assumptions about normal distribution will not hold here though)

Regarding the time series I would start even simpler. Depending on the number of customers, I'd start by visualizing some or all historical data in a line plot (sales per customer over time) and check the min, max and mean per customer. This gives some intuition regarding potential trends. For example, if all observations remain constant over time for a given customer, whether there is an upward/downward trend or if the data has high variance with no clear trend. Also, there may be clusters of customers which show similar patterns. Obviously, this is neither Machine Learning nor a rigorous statistical analysis but rather a pragmatic approach supported by some basic data analytics.

What you need to be very careful with is the time horizon of any quantitative prediction: based on 10 observations at $t \in \{1,21,...,201\}$ you may derive some conclusions for, let's say, $t<=301$ (to make up a total out of the blue ballpark figure) but $t=621$ is very far in the future. Also, you need to keep seasonality in mind. For example: If your observations are all from October to April of a given year and assuming you have a winter/summer seasonal pattern. Then you cannot infer a lot for months May to September. To understand the limitations and forecast potential of your time series better, I'd speak to subject matter experts, e.g. in sales & marketing. It could also be helpful to understand their forecasting approach and cross check any insights you derive with their predictions.

But, as Erwan pointed out, be very careful to derive conclusions. Applying some "ML magic" will not find a useful pattern if there insufficient data to find any signal. And of course additional data collection would be reasonable if that is an option.

$\endgroup$
2
  • $\begingroup$ Hi, thanks upvoted.. Can you also share your views on the timeseries part of the question? $\endgroup$ – The Great May 9 at 15:26
  • $\begingroup$ @TheGreat I have edited my answer and added some ideas on that. Please note that these are based on my experience of managing data analytics projects since that's how would approach this. It's not really an ML approach due to the data limitations. $\endgroup$ – Sammy May 9 at 20:40
1
$\begingroup$

The main problem with very little data is that it's almost impossible to know how representative the sample is. Some people would even say that less 20-30 data points cannot be representative of anything. Every single data point can have a huge impact on any model, so any prediction has a huge margin of error.

If one is going to train a model from a tiny sample, they should do everything possible to avoid overfitting: use very few features, use a very simple type of model with as few parameters as possible. Anything else is guaranteed to overfit.

In your example the behaviour of a customer can be very complex in reality, so it's particularly challenging (and potentially risky) to rely on such a small sample. Imho the first task is borderline but it can be tried, but I don't think the second one is realistic: revenue is not always regular, predictions in the long term are going to be barely better than random.

$\endgroup$
1
  • $\begingroup$ Hi, thanks upvoted.. So, you suggest that making any predictions using timeseries is useless and nothing better than a random guess. I get it. $\endgroup$ – The Great May 9 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.