I am in the following context:
Data: static, baseline health data at the patient level, 40 features, sparse (~ 25 binary features with many 0 or many 1 + other categorical features)
Objective : to cluster instances into clinically meaningful sub-population or clinical context, to get a sense of at risk sub-populations (on a Follow-Up outcome)
Considered approach (see this short blog article (2 minute's read) for its rationale):
- Fit a random forest on the Follow-Up outcome, using all features (no denoising or removing correlation)
- Use co-occurence in trees leaves to get a similarity matrix of the patients
- Turn in into a distance matrix
- Cluster patients with this distance matrix
My questions are as follow:
- I have only found literature using this methodology for unsupervised clustering (i.e. RF is learned on random target variable) : Shi and Horvath 2016 and Dalleau 2018 (unpublished). Does anyone have insights on those methods in general (other references, personnal experience...)?
- Do you know of any article written about the supervised use of RF to create similarity matrices (with or without mention of clustering)?