Prediction of a dependent variable having comma separated values

Question

1. I have a dataset from survey results of Genetic Testing that looks like below:

For more details sample csv file here:

https://drive.google.com/open?id=0BzFPqeNZO-6mM2VwZ1pRQ3NxYTA

I am trying to create a model to predict the column “actionspost” (the extreme right column of the above file). The column currently holds comma separated values like “2, 4, 6”. Each numerical value is a coded action that a patient would take after knowing the results of the genetic test as per the survey filled by the patient. For example, 2 indicates: Change eating habits, 4 indicates: Getting members of his/her family tested and so on. Based on the features in the csv file above, I want to create a model that would help me predict the “actionspost” values.

One way I thought, I could do this was break down comma separated values in “actionspost” to create duplicate rows for same customer Ids holding only one “actionspost” value at a time (For example: There would be 4 rows for customer ID: C00003 with first row having “actionspost” value of 2, second row with “actionspost” value 8, third with 9 and fourth with 10. Then create subset of the data set for each unique value of “actionspost”. This time, the dependent variable will not be actionspost but something like “isactionpost2?” (Yes =1 and No = 0). And train models for each of this subset for different values within the “actionspost” column.

I personally don’t like the above method as it too cumbersome and in my opinion, not the optimal one too. I was wondering if there could be a better way to address such scenario? My end goal is to try to train different models like Decision Trees, Naïve Bayes, and Neural Nets and check which one leads to better predictability.

Answer 1

Your problem is called multi-label classification. If you search for this term you will find a lot of literature. In short, these are the most common approaches:

• Treat each label independently and build a binary classifier per label (baseline approach, Eric Lecoutre's answer)
• Predict the power set, means create a target variable that represents all possible combinations and predict it using a multi-class classifier (Upper_Case's answer)
• Use a classifier that can deal with multi-labels inherently, e.g. a neural net with multiple output nodes and cross-entropy as loss function, or respective multi-label variants of decision trees or SVM. For example, scikit-learn supports multi-label implementation for decision trees.
• Classifier chains: Train a binary classifier per label, but use the predictions of one classifier as input to another classifier

Depending on the nature of the data, one or the other approach will work best. Start with a binary classifier per label to get a baseline. If the labels are correlated (some labels frequently occur together), there is potential for improvement using one of the other methods. Using one of the existing implementations of multi-label classifiers is probably the easiest next step. The power set approach only makes sense if the number of possible combinations is reasonable. I am not sure if classifier chains are worth the effort, if you are not doing this for a Kaggle competition :)

As always, setup a proper cross-validation strategy, define the evaluation metric (e.g. micro/macro f-measure or cross-entropy loss) so that you are able to identify the best model.

Answer 2

The easiest approach I can think of would be to recode the actionspost values to a multidigit set of binary flags. So if you have 15 possible responses, and the subject either took an action or did not, this would be a 15 digit number with each digit representing an action that was taken (=1) or not (=0).

This would let you investigate individual actions, every observed combination of actions, as well as number of actions taken very easily without any row duplication.

Answer 3

You are in the right direction. Indeed, actions must be considered as distinct possible outcomes -> you should create one model per type of action.

In terms of data management, what you propose may be dangerous, please directly think about the desired outcome: n different training datasets. If you indicate which statistical environment you intend to use, we can propose best practices to code.

By the way, in such situation, trying to predict number of reactions or simply is there a reaction (yes/no) is also of interest.