5
$\begingroup$

I have some JSON data to be transformed to a machine-learning friendly format. Every object in my data, which will eventually become an instance in my dataset, has the exact same fields (in this case, foo, bar and array). The array field contains a variable number of sub-objects (from 0 to 10). Each of these sub-objects has one categorical field with a massive range and some other fields which can be safely ignored. All categorical fields in these sub-objects belong to the same range.

Example (massively simplified object):

{
"foo": 1
"bar": 0.5
"array" : [
    {
        "categorical": "Lorem"
        "other": 34
        "stuff": 56
    },
    {
        "categorical": "Ipsum"
        "other": 53
        "stuff": 12
    },
    {
        "categorical": "Dolore"
        "other": 6
        "stuff": 101
    }
]
}

Obviously foo and bar are easily represented as numeric attributes. I would now like to represent this array (array) as one large one-hot (or several-hot) vector. Assuming I don't care about any other field of the sub-objects except for the category field categorical, here is my question:

Question

Is it valid and possible to set more than one bit in a one-hot vector (which would make it, I assume, a several-hot vector) to represent all categories present in this instance? If not, how could it be done?

Improve this question
$\endgroup$
3
  • $\begingroup$ I am not sure I understand the data to answer intelligently. Each instance always has foo and bar and then n categorical variables? What do other and stuff represent? Can you make up a representative set of data for another domain? $\endgroup$
    – CalZ
    May 24, 2017 at 0:55
  • $\begingroup$ Sure, I will clarify all of this in the post, thank you for pointing it out. Unfortunately I cannot divulge the real dataset, but my question is more conceptual than context-specific. $\endgroup$
    – Paul Benn
    May 24, 2017 at 10:52
  • $\begingroup$ this is not a problem at all. Bag of words is a classic example of this. use a model that can handle sparse matrices ( to save memory) and use regularisation eg l2 or l1 to reduce impact of instances with very few examples $\endgroup$
    – seanv507
    Jul 23, 2017 at 20:39

1 Answer 1

Reset to default
1
$\begingroup$

Yes, it will work. Basically by creating the encoding, scikit-learn's label encoder does the same thing, you are creating such more features each representing the presence or absence of that level in your sample. To represent d levels you need d-1 variables, so in your case that is what is exactly happening. However since you said, the number of levels can be massive it will not be a good idea to use a vector that massive. It would be a good idea to run some preliminary analysis and check if some levels are totally useless and dont relate to the response as well, some might have the exact same effect and hence those levels can be combined.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I see, fair enough, I had been thinking of running some form of PCA on this. Thank you for the clarification, can you link me to any sources that prove this or is it trivial? Also, as a follow-up, what if I did care about the other stuff within the sub-objects? Would the best approach be to hash the values and add the hash to the categorical value, thus creating even more dimensions? Sorry for all the questions, I am very new to ML. $\endgroup$
    – Paul Benn
    May 24, 2017 at 14:38
  • $\begingroup$ So your features for your above sample are as follows: foo, bar, other, stuff, then categorical which will be the number of levels you have so if ten levels then ten added features for those. It is trivial, just dont think of categorical as one feature , every level is a separate feature. If you google how to handle categorical variables with multiple levels, it will lead you to some good blogs on best practices for handling such situations. $\endgroup$
    – Himanshu Rai
    May 24, 2017 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.