This method is pretty much time consuming, but you may reduce the database useing sklearn train test split to get, say, 10% extract.
AB = A blue
AO = A orange
BB = B blue
Spoiler: in the end you will get several classes, they are the different clusters.
The idea is as follows.
Put all clusters into one class: C1 = [AB, AO, BB, BO, CB, CO]
Start iterating through your data, get the next element X.
Check if X in each element of class behaves the same way - either in or out.
Split the classes that behave different. For example, if you check a point in the upper left corner, [AB, BO, CO] will say YES, whereas [AO, BB, CB] will say no. So we have two classes C1 = [AB, BO, CO] and C2 = [AO, BB, CB]
Continue (go to step 3) until either each cluster is in a separate class or there is no more data.
So at some point, BO and BB will be in separate classes, but AB and CO will always behave the same way.
Here I am inspired by the algorithm that minimizes a DFA (Deterministic Finite Automaton).