I was thinking about this lately. Let's say that we have a very complex space, which makes it hard to learn a classifier that can efficiently split it. But what if this very complex space is actually made up of a bunch of "simple" subspaces. By simple, I mean that it would be easier to learn a classifier for that subspace.
In this situation, would clustering my data first, in other words finding these subspaces, help me learn a better classifier? This classifier would essentially be an ensemble of each subspace's classifier.
To clarify, I don't want to use the clusters as additional features and feed it to a big classifier, I want to train on each cluster individually.
Is this something that's already been done/proven to work/proven to not work? Are there any papers on it? I've been trying to search for things like this but couldn't find anything relevant so I thought I'd ask here.