# Random Forest Regression. How to represent really long list of categories for processing

I'm trying to build a model to solve a regression task.

Simplified, the data look like:

7.16, A 1, B 4, C 15, D 3
8.21, C 3, D 2, G  7, M 4, Y 9
1.85, D 3, N 1, L  1


The first column is a label, consequent columns are a class and respective number.

For example, the first row demonstrates that there is one of A, four of B, fifteen of C, and three of D - such a configuration gives us 7.16.

My Primary Question:

How do I represent my data so that I can fit a Random Forest Regression model?

My Thoughts:

1. Figure out the whole list of classes (there are 26 letters, but I have way more classes) and put zeros for all of the unknown classes into each record. Vector.sparse() might help here as it can reduce the amount of required memory.

2. Figure out the entire list of classes, correspond a number starting from 0 and convert each record into a vector (i.e. the first one would look): 0 1 1 4 2 15 3 3. Then each column is treated as a separate feature, but in this case I have to specify that some of my features are categorical and that's not possible with Spark since different records are of different length (In Spark I have to provide columns of features, which are categorical with number of classes. In the example that would be a map(0 -> 26, 2 -> 26, 4 -> 26) - like every second column is a class with 26 categories).

3. It's possible to build features like A:1, B:4, C:15, D:3 and treat all of them as categorical. Or move decision burden (categorical vs continuous) to Spark itself.

• Why not have the inputs be $\mathbb R^{26}$ vectors like $x_1 = (1,4,15, 3, 0, \dots)$? The unmentioned columns are presumably zero?
– Emre
Dec 14 '15 at 17:16
• I think there needs to be more consistent terminology for this to be clearly articulated. Generally "classes" are categorical outputs for a model. I think hear you are saying there are many features (which for now you have labeled A, B... Z), and you have counts of each feature. If the numbers here are not counts, but rather something categorical as well, then your second approach makes more sense. More information about what each of these features (A-Z) means would help. Dec 14 '15 at 18:17
• @jamesmf that's correct - these are features (ABC...Z) and numbers are their counts. It's a description of a molecules. There are many molecules in one file. Each one has several compounds with their respective count. F.e. one of molecule has "C10C1007 3 C10C1012 4 C40C3104 1 Cl10C1004 8" etc. Total number of features is 119. The file I have (160 MB) contains overall 7700 unique compounds (features). I've seen 8+ GB file, although I'm not sure if number of compounds (features) there much bigger. Dec 14 '15 at 18:53
• So you have molecules, each of which has 'sub-graphs' (features) C10C1007, C10C10112... and the number following that is a count, right? So if a molecule has 2 sets of C10C1007, I'd see "C10C1007 2" correct? And there are 119 of these features, with 7700 rows, or molecules? Dec 15 '15 at 15:58
• @jamesmf partially true. I have a file with 37291 molecules. These molecules have features like C10C1007, C10C10112, etc. For each feature there is a corresponding count. The first molecule only has 119 such a features. Overall, for all molecules in the file, there are 7700 unique features. Dec 15 '15 at 16:10