I trained an image auto-encoder on a large dataset, and now have for every image, an n-dimensional feature vector. This vector is not spatially correlated to the image. I now used this embedding space directly for classification, using fully connected dense layers. I have 55 classes, and the vector dimensionality is 1024. Even a single classification layer on top of the embedding, means 1024*55 parameters and it very easily overfits.
Since my embedding is not spatial in any sense, does it make sense to use Trees for classifying over the embedding space? I.e, feed the feature vectors to a tree based classification method, instead of a fully connected network. Are there cases, depending on the data, where trees outperform neural networks. If yes, what are such cases?
