0
$\begingroup$

I have 1 year transaction level data aggregated at a weekly level for 1000 different stores. I want to cluster similar stores based on 8 variables such as sales, customer count etc. The concern is that as the data is at a store week level I cannot directly run the clustering algorithm as it may compare and classify the data points from same store into one cluster, which is not what is required.

So the second option is to transpose the data i.e. 1 data point for each store having 8(variable) * 52(week) columns and then run clustering on it. I am not sure if k-means or the converntional clustering method will work well on such high dimensional data.

So I am looking for suggestions as to how can we handle this situation.

$\endgroup$
1
$\begingroup$

You basically have data with a panel structure ($i=$shops, $t=$ weeks). So for a simple clustering problem, you need to "tell" the algorithm that observations for each $i$ (shops) are not independent.

I think you are on the right way to treat each week $t$ as an own variable/feature here. In this case you would have 416 features and 1000 observations. This should be "okay", but especially with non-supervised problems, it is up to a try.

In any case, I would also try to do a little aggregation on the data (over time). Maybe there are good reasons to do this by month or quarter (or some other sort of "season"), meaning to sum up the weekly values over some period of time. Say you do this for quarters, you would have "only" 32 features, which should be absolutely fine with 1000 observations.

See also this post: https://stats.stackexchange.com/questions/88320/cluster-analysis-on-panel-data

$\endgroup$
  • $\begingroup$ I thought about aggregating it over a monthly level but I am a bit skeptical that this level of aggregation may hide some patterns in the data. But yeah I will try that if the weekly results don't look ok. $\endgroup$ – bakas Sep 25 at 2:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.