How to run KNN (or other) on nested features? Image metadata

Question

I have looked everywhere and I can't find a straight solution. I have a set of metadata from images and their elements (name, heigh, width, position{top, right}). I would like to create categories of images based on relative positions of several items in the image.

Example:

img1 has 

logo in top x1, right y1, and is height h1 and width w1

text in top x2, right y2, and is height h2 and width w2

from img1 to n

I thought about running a KNN classifier to find images that are more similar to others but I am having trouble thinking about how to organize the data.

Should I group them by image? In which case: how would KNN or other algorithms work on grouped data? Should I do OHE to know if there is a specific element? Would it be better to use a DecisionTree?

Its my first data science job, and there is nobody with seniority here so I am having trouble organizing this dataset.

Thanks in advance!

Answer 1

You could try to apply a dimension reduction technique to map all of the variables into 2, such as t-SNE or UMAP (I recommend this one), and then apply a clustering algorithm based on densities, such as HDBSCAN.

Be mind that each run of the HDBSCAN might return different results. Scale all the variables before starting to apply the algorithms (scale)

Links to these algorithms in R: UMAP: HDBSCAN