5
$\begingroup$

I'm trying to build a model to solve a regression task.

Simplified, the data look like:

7.16, A 1, B 4, C 15, D 3
8.21, C 3, D 2, G  7, M 4, Y 9
1.85, D 3, N 1, L  1

The first column is a label, consequent columns are a class and respective number.

For example, the first row demonstrates that there is one of A, four of B, fifteen of C, and three of D - such a configuration gives us 7.16.

My Primary Question:

How do I represent my data so that I can fit a Random Forest Regression model?

My Thoughts:

  1. Figure out the whole list of classes (there are 26 letters, but I have way more classes) and put zeros for all of the unknown classes into each record. Vector.sparse() might help here as it can reduce the amount of required memory.

  2. Figure out the entire list of classes, correspond a number starting from 0 and convert each record into a vector (i.e. the first one would look): 0 1 1 4 2 15 3 3. Then each column is treated as a separate feature, but in this case I have to specify that some of my features are categorical and that's not possible with Spark since different records are of different length (In Spark I have to provide columns of features, which are categorical with number of classes. In the example that would be a map(0 -> 26, 2 -> 26, 4 -> 26) - like every second column is a class with 26 categories).

  3. It's possible to build features like A:1, B:4, C:15, D:3 and treat all of them as categorical. Or move decision burden (categorical vs continuous) to Spark itself.

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ Why not have the inputs be $\mathbb R^{26}$ vectors like $x_1 = (1,4,15, 3, 0, \dots)$? The unmentioned columns are presumably zero? $\endgroup$
    – Emre
    Commented Dec 14, 2015 at 17:16
  • $\begingroup$ I think there needs to be more consistent terminology for this to be clearly articulated. Generally "classes" are categorical outputs for a model. I think hear you are saying there are many features (which for now you have labeled A, B... Z), and you have counts of each feature. If the numbers here are not counts, but rather something categorical as well, then your second approach makes more sense. More information about what each of these features (A-Z) means would help. $\endgroup$
    – jamesmf
    Commented Dec 14, 2015 at 18:17
  • $\begingroup$ @jamesmf that's correct - these are features (ABC...Z) and numbers are their counts. It's a description of a molecules. There are many molecules in one file. Each one has several compounds with their respective count. F.e. one of molecule has "C10C1007 3 C10C1012 4 C40C3104 1 Cl10C1004 8" etc. Total number of features is 119. The file I have (160 MB) contains overall 7700 unique compounds (features). I've seen 8+ GB file, although I'm not sure if number of compounds (features) there much bigger. $\endgroup$
    – evgenii
    Commented Dec 14, 2015 at 18:53
  • $\begingroup$ So you have molecules, each of which has 'sub-graphs' (features) C10C1007, C10C10112... and the number following that is a count, right? So if a molecule has 2 sets of C10C1007, I'd see "C10C1007 2" correct? And there are 119 of these features, with 7700 rows, or molecules? $\endgroup$
    – jamesmf
    Commented Dec 15, 2015 at 15:58
  • $\begingroup$ @jamesmf partially true. I have a file with 37291 molecules. These molecules have features like C10C1007, C10C10112, etc. For each feature there is a corresponding count. The first molecule only has 119 such a features. Overall, for all molecules in the file, there are 7700 unique features. $\endgroup$
    – evgenii
    Commented Dec 15, 2015 at 16:10

2 Answers 2

Reset to default
1
$\begingroup$

If the problem is that you don't know all the categories in advance and different records have different categories, then the hashing trick might help.

Improve this answer
$\endgroup$
0
$\begingroup$

I think that your first and last approach are the same - as a result you have sparse feature vector. In your last approach you just don't list features with value 0.

It is hard to say if you should use categorical or continuous values - it depends on each feature.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Although they're in a way similar, I'd say the first case allows me to use continuous features as the last one treat all of them as categorical. It might have implications on the algo's performance in terms of accuracy or speed, or both. $\endgroup$
    – evgenii
    Commented Dec 14, 2015 at 18:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.