Stats version: I have a few measurements of a function that takes three inputs and produces a few 2D fields of outputs: $f_i(a,b,c;x,y)$, with $f$ being a vector of several quantities. I would like to cluster the points in $(x,y)$-space into regions that have a similar response to the input variables $(a,b,c)$. If I treat the points independently, applying a clustering algorithm is fairly easy. (I can elaborate on which one I'm using if you think it's relevant, but I'm open to suggestions on your favorites.) But there's no guarantee that the clusters are near each other spatially. Having contiguous clusters of roughly equal size is definitely a plus. What are some ways to modify the approach to add in that constraint?

Science version: I'm running an atmospheric chemistry model and varying emissions over the US. In that setup, $(a,b,c)$ are the national-total emissions of NOx, SO2, and NH3; $(x,y)$ are grid points over the US; and $f_i$ is a vector of the various chemical species I'm tracking. I've varied $(a,b,c)$ in a number of simulations. Now I want to divide the country into regions that have broadly similar responses to the emissions changes, and create simplified models for a manageable number of regions. I'm particularly interested in nonlinearities/mixed-effects/second-order derivatives in the chemistry response. It's possible that different regions of the country do respond similarly to emissions even if they're not near each other, but the different regions should have some sort of spatial agglomeration. Thoughts?

Side question: I originally played around with this a few years ago with Matlab code but have since migrated to Python. Suggestions on how to efficiently use Python stats packages for this sort of problem?

  • $\begingroup$ If you are coming from matlab, numpy will feel familiar. Most python data science packages are built off of it. Pandas can be nice for reading in data (in memory), and scikit-learn has an implementation of many ML algorithms and models. $\endgroup$ – jamesmf Nov 5 '15 at 16:28

I'm not sure I'm following your question completely, but for spatial type clustering maybe a Self-Organizing Map would meet your needs. Basically it maps out the space based on nearby activations.

Side answer: I haven't used python

| improve this answer | |
  • $\begingroup$ Thanks for the tip, I'll check it out! It looks like it'll help. The difficulty is going to be in defining a good metric for "similarity of response to inputs" in any case. $\endgroup$ – Jareth Holt Jul 7 '15 at 23:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.