Classify Customers based on 2 features AND a Time series of events

Question

I need help on what should be my next step in an algorithm I am designing.

Due to NDAs, I can't disclose much, but I'll try to be generic and understandable.

Basically, after several steps in the algorithms, I have this:

For each customer that I have, and events that they do during a month, during first steps I have clustered the events into several categories (each customer will have the events separated into categories that go from 1 to x being x between 1 to 25, generally the first categories have more density of events than the others).

For each category and customer I have created a time series aggregating the events of the month per hour (getting patterns of when these events are being done). Also I am using a couple of normalizing variables based on the number of days over a month (30 days) that the guy performs at least one event, and the number of days with at least one event over the total of days with at least one event (aggregating all clusters). The first one gives me a ratio of how active the customer is during the month, and the second one weights the category against the others.

The final table looks like this

|*Identifier*|  *firstCat* | *feature1* | *feature2*  |   {      *(TIME SERIES)*   }

CustomerID  |  ClusterID |  DaysOver30 | DaysOverTotal | Events9AM Events10AM ... 

 xx | 1 | 0,69 |  0,72 |  0,2   0,13   ...

 xx | 2 | 0,11 |  0,28 |  0,1   0,45   ...

 xy | 1 | 0,23 |  0,88 |  0,00  0,60   ...

 xy | 2 | 0,11 |  0,08 |  1,00  0,00   ...

 xy | 3 | 0,10 |  0,04 |  0,40  0,60   ...

The time series variables are the percentage over the total of events per day on each specific category (this means that per each row adding up all variables should be 1). The reason of doing it like that is because for example a time series with events 0 0 0 1 0 and 1 1 1 2 1 are completely different, and standardizing to normal would give similar results. And due to high skew between different categories, I check the values on the time series independently with the others.

What I need to do now is to identify these categories (remember, they can be from 1 to x being x any number from 1 to 25) into 3 tags: tag A, tag B and None of Them. Looking at these variables I can manually identify which tag they belong to, and the idea is to identify manually as much as I can and use any classifier algorithm to learn from that and identify all of them.

My idea was to use multiple logistic regressions on the table, but all the variables of the time series are correlated (since they are a linear combination of each other), so I thought I better use a clustering algorithm only over the time series using euclidean distance to categorize the different patterns and use the result and the other two normalizing variables in the logistic regression.

The other concern that I have is that this approach takes each row independently from the others, and in theory, for each customer there should be only 0 or 1 tag A, 0 or 1 tag B and the remaining of them should be None (another tip is that normally Tag A and B are between first categories, because is highly dependent on the normalizing features (if days over total is High, there is a high possibility that the row is either A or B, depending on the Time Series Pattern).

Edit: This is no longer a concern, I will just perform two different logistic regressions, one for Tag A or Other and another for Tag B or another, with the result probabilities I can select only the best of each.

The dataset is enormous and the final algorithm needs to be applied using SQL (on Teradata), but for getting the coefficients of the logistic regression, or the centers of the clustering I get a sample and use R.

Answer 1

I know it has been a while since the question has been posted but for future readers I propose here another solution for the problem of classifying time-series of proportional data i.e., vectors of proportions.

Hidden Markov Models (HMMs) are broadly used for time-series classification. The original implementations were developed for discrete and Gaussian data but since then, they have been extended to more probability distributions.

Among them, the Dirichlet, the generalized Dirichlet, and the Beta-Liouville distributions which are perfectly suited to work with proportional data.

Usually, a model is trained for each category ("Tag" in this question) from time-series for which the category is known. When a new time-series has to be classified, its likelihood with respect to each model is computed. The new time-series is usually assigned to the category leading to the highest likelihood.

Here are the reference papers for these models presenting them more deeply and providing all the equations needed for implementation:

• Dynamical Dirichlet Mixture Model, Research Report from IDIAP, L. Chen, D. Barber, J.-M. Odobez, April 2007
• Proportional data modeling with hidden Markov models based on generalized Dirichlet and Beta-Liouville mixtures applied to anomaly detection in public areas, E. Epaillard, N. Bouguila, Pattern Recognition 55, pp 125-136, 2016

For the additional two features mentioned in the question, HMMs for mixed data have also been developed in the following research paper: Hybrid hidden Markov model for mixed continuous/continuous and discrete/continuous data modeling, E. Epaillard, N. Bouguila, MMSP, pp. 1-6, 2015.

These models are heavier to deploys than regression models but have the advantage of being discriminative and generative models and to be able to generalize well when some variability is present in the processed data.