5
$\begingroup$

I have a dataset that has (among others) a categorical variable with many levels and further attributes associated with each level.

For example, consider predicting machine failure based on its last repair report.

  • A machine has many (many) different parts.
    • Every part could have been damaged or not.
      • If damaged, it could have been inspected, repaired or replaced etc.
        • These operations have associated costs and the part itself has a money value and other attributes (like the time it took to repair it...).

If I want to make predictions on the level of a machine, I need to aggregate all this information. I could one-hot encode all the parts indicating whether they have been damaged or not. But this still leaves me with the other attributes on the lower part level, like price and performed operation. I could probably further expand them into individual columns by considering all the combinations like part_X-repaired-..-price, part_X-replaced-..-price, ... part_Z-replaced-..-price but this seems to get out of hand.

Is there a better way to handle this type of data? I was thinking maybe some clustering technique but when I try to set it up I run into the same problem.

It is somewhat a reversed hierarchical model structure (in hierarchical (linear) models the outcome variable is on the lowest level if I'm not mistaking).

Improve this question
$\endgroup$
1
  • 3
    $\begingroup$ The most obvious formalism is a graphical model, but they're a bit involved and scalability can be an issue. There's a great course on Coursera if you're not familiar. For implementation you should look into a new library called edward. If that sounds like too much work, just expand the tree like you suggested. $\endgroup$
    – Emre
    May 25, 2017 at 22:55

2 Answers 2

Reset to default
2
$\begingroup$

This is called "bilevel optimization" where one problem is embedded (nested) within another. If there are more than one objectives at one or both levels, it is then called "multi-objective bilevel optimization". There is a rich literature on this problem, including the book Multilevel Optimization: Algorithms and Applications.

Finding solutions to minimize the objectives often does not require machine learning, just linear programming.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your answer. If the "outer" problem is predicting machine failure, what would the embedded problem be? Not quite sure that's the right tool here... $\endgroup$
    – oW_ ♦
    Nov 13, 2017 at 16:44
  • $\begingroup$ The embedded problem would be predicting part failure. Some parts might be shared across machines then you could attribute aspects of machine parts to specific part failure patterns. $\endgroup$
    – Brian Spiering
    Nov 18, 2017 at 17:01
1
$\begingroup$

I could probably further expand them into individual columns by considering all the combinations like part_X-repaired-..-price, part_X-replaced-..-price, ... part_Z-replaced-..-price but this seems to get out of hand.

That is the way I would go if the approach you are using doesn't like high cardinality categorical variables. There might be some interesting relationships that fall out, like part A tends to need a repair when part B is replaced with a cheap version. If you end up with too many variables, you could try something like PCA to reduce them.

[Edit] Thought of another thing to look into: Vowpal Wabbit. It is an algorithm targeted almost exactly at your problem - lots of sparse variables. They claim it can handle 1 billion.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ From my understanding, the problem involves machine parts which are discrete. PCA is not the appropriate tool for discrete-valued dimension reduction. Multiple correspondence analysis is more useful for discrete-valued dimension reduction. $\endgroup$
    – Brian Spiering
    Dec 13, 2017 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.