Clustering stores based on weekly data

Question

I have 1 year transaction level data aggregated at a weekly level for 1000 different stores. I want to cluster similar stores based on 8 variables such as sales, customer count etc. The concern is that as the data is at a store week level I cannot directly run the clustering algorithm as it may compare and classify the data points from same store into one cluster, which is not what is required.

So the second option is to transpose the data i.e. 1 data point for each store having 8(variable) * 52(week) columns and then run clustering on it. I am not sure if k-means or the converntional clustering method will work well on such high dimensional data.

So I am looking for suggestions as to how can we handle this situation.

Answer 1

You basically have data with a panel structure ($i=$shops, $t=$ weeks). So for a simple clustering problem, you need to "tell" the algorithm that observations for each $i$ (shops) are not independent.

I think you are on the right way to treat each week $t$ as an own variable/feature here. In this case you would have 416 features and 1000 observations. This should be "okay", but especially with non-supervised problems, it is up to a try.

In any case, I would also try to do a little aggregation on the data (over time). Maybe there are good reasons to do this by month or quarter (or some other sort of "season"), meaning to sum up the weekly values over some period of time. Say you do this for quarters, you would have "only" 32 features, which should be absolutely fine with 1000 observations.

See also this post: https://stats.stackexchange.com/questions/88320/cluster-analysis-on-panel-data