4
$\begingroup$

I am in the following context:

Data: static, baseline health data at the patient level, 40 features, sparse (~ 25 binary features with many 0 or many 1 + other categorical features)

Objective : to cluster instances into clinically meaningful sub-population or clinical context, to get a sense of at risk sub-populations (on a Follow-Up outcome)

Considered approach (see this short blog article (2 minute's read) for its rationale):

  1. Fit a random forest on the Follow-Up outcome, using all features (no denoising or removing correlation)
  2. Use co-occurence in trees leaves to get a similarity matrix of the patients
  3. Turn in into a distance matrix
  4. Cluster patients with this distance matrix

My questions are as follow:

  1. I have only found literature using this methodology for unsupervised clustering (i.e. RF is learned on random target variable) : Shi and Horvath 2016 and Dalleau 2018 (unpublished). Does anyone have insights on those methods in general (other references, personnal experience...)?
  2. Do you know of any article written about the supervised use of RF to create similarity matrices (with or without mention of clustering)?
Improve this question
$\endgroup$
2
$\begingroup$

I recently presented a poster at a conference where we used the same approach you describe for clustering. Generally, I think it's a great approach for clustering, as you can get clusters which have a distance that is relative to an outcome or variable that you are interested in.

For some insight, I have a few pointers: 1) When getting the co-occurence in trees, depending on how many subjects you have, this can end up being a very sparse matrix. To make the matrix less sparse you can increase the minimum number of samples required in terminal nodes. 2) After you get the similarity matrix, do a PCA on this, ending with individuals in rows and PCs in columns. Get the distance between individuals in this PCA space, restricting to some top number of components that you find acceptable. I recommend this because the similarity matrices can be huge if you have a lot of cases.

Improve this answer
$\endgroup$
  • $\begingroup$ Would you mind sharing references of this poster (if public)? I am curious of effectiveness in practical applications. I started a small python module to create this kind of similarity matrix and show the results using MDS. Adding the PCA step could be a good way to reduce the final matrix and ease further clustering. $\endgroup$ – CharlesG Aug 21 '19 at 8:17
0
$\begingroup$

The original book on random forest discussed unsupervised use and proximities.

But I don't know if that is all as shiny as advertised. I am not aware of any prominent use of these techniques except of variable selection.

Improve this answer
$\endgroup$
  • $\begingroup$ This is more of a short description of a method than a study of its behavior. I am neither aware of any work using this technique, but a few feasibility tests shows some potentiality. I should publish some notebooks about that. $\endgroup$ – CharlesG Apr 22 '19 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.