I'm working on one use case where I have to explore source code repo files. Different files will be a categorical values for me. But with such large number of files, One Hot Encoding comes out to be very large.

Also, all files are divided among unique modules, such that each files belongs to a specific module. So if I consider unique files associated with specific modules comes out to be small in number and One Hot Encoding matrix is also not large.

  1. So if I define different modules as a categorical features, I'm not sure how to map the associated unique files.
  2. How to derive new feature with such combination of two categorical features?
  3. Any other better way to handle such use case?

1 Answer 1


Recently practitioners are representing categorical variables as embeddings for ML models. I can see a solution to your problem there.

As your problem is having a two-level hierarchy you can consider two embeddings, one set of embeddings for modules, and another set for files. For every document, you can take their combination and pass it as input to the model. In this way, both the module and file-level information will be captured.

References: https://towardsdatascience.com/deep-embeddings-for-categorical-variables-cat2vec-b05c8ab63ac0

  • $\begingroup$ Thanks for your response. If I have to use embedding, then I think only one embedding matrix is also sufficient. I will feed Module+FileName combination as an Input. Do you think any other approach as well? As I dont want to train another model. $\endgroup$ Commented Oct 9, 2020 at 8:45
  • $\begingroup$ Using one embedding matrix will work fine until you introduce a new file in an existing module. In this case, you are forced to take a random vector for both file and module, but if you use two embedding matrices you can reuse the module's embedding. $\endgroup$ Commented Oct 9, 2020 at 13:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.