# Multi-label classification with nested features

I need to perform a multi-label classification. I have three features and they are nested. I am unsure how to combine this or what kind of classification algorithm would be best. Some multi level neural network as shown here seems good, but the nested features don't seem to be taken into account there.

I present the nested features (X) and labels (Y) in the two datasets below: one subject ID can have one or more features and one or more classes. Features and classes can be 'occupied' by one or more subject.

Note: I have about 100k subjects, 1k features (at the third level) and 200 classes.

data_features
subject_id   feature1   feature2   feature3
1          a         aa          aaa
2          a         aa          aab
3          a         ab          aba
1          a         ab          abb
2          b         ba          baa
3          b         ba          bac
3          c         ca          caa
4          c         ca          caa
5          c         cb          cba
6          c         cb          cbb

data_labels
subject_id   label1   label2   label3   label4
1        0        1        0        0
2        0        1        1        1
3        0        1        1        0
4        1        1        0        1
5        1        0        0        0
6        0        1        1        1
7        0        0        0        1
8        1        1        1        1
9        0        0        1        1
10        1        0        1        0
11        0        1        0        1
12        1        0        0        1


I am quite unsure what algorithm would combine those the best? (I am skilled in R and SAS and decent in Python, but will learn any other language that would be needed)

• Can you explain why the features are "nested"? What do they represent, and what is the relationship between a feature and a sub-feature? In other words, what would be the problem if all the features are "flatten", i.e. taking all the sub-features as if they are regular features? Nov 15 '20 at 12:48
• If you know feature 3, you automatically know feature 2 and 1. If you know feature 2, you automatically know feature 1. That's what I mean by nested. Technically I could flatten and dichotomize everything, but it would give me a dataset with 100k rows and 2k columns. That's some heavy lifting neural network. I was hoping that, due to the nature of the feature data, some simplification existed. Nov 15 '20 at 16:42
• "If you know feature 3, you automatically know feature 2 and 1" -> that would means that features 2 and 1 don't bring any additional information, unless there are cases where feature 3 is not provided but features 2 and 1 are? Nov 15 '20 at 17:40
• Technically they aren't but they may have prediction use. A class may for example be linked to feature2='aa'. But if you ignore feature2 and consider only feature3, the link may get 'lost' in noise, because that specific class has multiple possible feature3 results, and 'aaa' and 'aab' would be considered just as different as 'aaa' and 'ccc'. Nov 15 '20 at 17:48
• So if I understand correctly the relationship between a feature and its nested feature is from some kind of general information to more specific, right? Possibly something similar to feature 1 = brand , feature 2 = brand + model, feature 3 = brand + model + version ? Nov 15 '20 at 22:07