I am attempting to improve on a method which predicts heterogeneous values based on their Cartesian coordinates (x and y). The current method first clusters the training set values, then trains a GPR model for each cluster, where the inputs are the coordinates of each value in the cluster, and the targets are the corresponding values. The drawback to this method is that each regression model only trains on a subset of the training data (the clustered subset). My proposed improvement is to use fuzzy k-means, and for each cluster, scale each target in the training data by it's membership to the cluster, then train the entire training set.

What I'd like to know is if this scaling makes sense theoretically. I am concerned, because if I use the same approach to a hard k-means clustered set, the regression algorithm trains on skewed data (the targets belonging to a cluster are scaled by $1$, and everything else is zero). The matrices below might clarify my idea. If I have two clusters, I would multiply each target value element-wise by the first column, train the GPR, then repeat with the second column.

$$\begin{equation} U_{N\times C} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 0 \\ .. & .. \\ 0 & 1 \end{bmatrix} \quad U_{N\times C} = \begin{bmatrix} 0.8 & 0.2 \\ 0.3 & 0.7 \\ 0.6 & 0.4 \\ .. & .. \\ 0.9 & 0.1 \end{bmatrix} \end{equation}$$

EDIT: I'm not sure if this confirms my suspicions or invalidates my approach, but I have compared my method to the older method, and it turns out the older approach is a better predictor. Additionally, there seems to be very little difference between scaling the targets before training regressor, and scaling the predicted values after training on the unweighted targets. I'd appreciate any insights you might have.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.