Time Series Forecasting with Neural Network (mixed data types)

Question

I have a dataset with the following format:

TimeStamp  | Action  | UserId
2015-02-05 | Action1 | XXX
2015-02-06 | Action2 | YYY
2015-02-07 | Action2 | XXX
...

I try to forecast future Actions for specific users based on the Users history in the dataset. I started with ntstool in MATLAB, but it can't handle mixed data types or non-numerical values. Now I am looking for other methods to predict future Actions and find periodic patterns in the records.

Is it possible to convert the values for Actions to numeric values or is there a possibility to create a neural network with mixed datatypes on the input? Maybe in R?

Answer 1

I guess I answered the question in the comments, so here goes.

Most ML models cannot deal with categorical values. A common way to solve this is to use one-hot encoding, also known as dummy variables. For very possible value of your categorical variable you create a column which is 0 unless this row has this category, then it is 1. It is possible to remove one of the categories since it is a linear combination of the other dummy variables (if all are 0, the last one must be 1).

The downside of this method is that it increases the dimensionality of your feature space. If you have enough data to support this or not that many categories that is not a problem. There are other alternatives, like taking the average feature of every category and adding that to your features as opposed to the categorical feature.

Answer 2

You can convert categorical data into vectors using embedding layer before the neural network input layer. You can check this about how to use mixed data for Neural Networks. Check how LSTM, RNN networks consume text input as time-series of word vectors. Word-vectors are computed for each word at the embedding layer whose dimensionality you can control by defining the embedding layer.