I am performing analysis on the well-known 'Adult' data-set, available on UCI using Orange Data Mining. In a PhD thesis, Pelleg (2004; pg 79) uses unsupervised clustering of the prescribed training set, setting the number of clusters to 4, and then uses the resulting clusters to define continuous regions of feature space defined by each cluster. During clustering, the target variable, 'income' (two classes: '>50K', and '<=50K') is initially treated as one of the features. Each cluster region is then ascribed a class, depending on the majority target variable class, i.e., '>50K' or '<=50K'.
To evaluate the 'cluster training', he determines whether each instance in the independent test-set falls into cluster region (e.g., 1 = '>50K', 2 = '<=50K', 3 = '>50K', and 4 = '<=50K', for example). A confusion matrix can then be built from this simple exercise. Pelleg quotes an accuracy of up to 97%.
I tried training with decision tree, random forest, and kNN learners (testing and training on the same data) I ended up with 90% ACC (RF learner). I decided to give Pelleg's concept a shot. Right now, I am not concerned with achieving a high ACC, just with being able to build a training/testing workflow in Orange, effectively using clustering as the learner. I have managed to:
- randomly select 3000 instances (data sampler widget) from the training set (k-Means clustering can take a max of 3000 instances)
- convert the target variable to a feature variable ('select columns' widget)
- use k-Means clustering widget, setting 4 clusters
- convert the cluster meta column of the training set to the target variable ('select columns' and 'edit domain' widgets) by setting the majority target class in each cluster to the whole cluster.
- Feed in the clustered training set and original test set to the 'test and score' widget, using kNN as the learner.
Steps 4-5 was the only way I could figure to determine if each test instance falls the 'region' of each cluster (i.e., using kNN learner to find the nearest training instance and effectively placing the test instance in that cluster).
In Orange Data Mining, is there a better way to establish/define continuous cluster 'regions' which can be labeled as a target class (perhaps weighting the size of the region by the size of the cluster?)
Can those regions then be used to test and score/evaluate/build a confusion matrix?