# Advise on making predictions given collection of dimensions and corresponding probabilities

I am a CS graduate but am very new to data science. I could use some expert advise/insight on a problem I am trying to solve. I've been through the titanic tutorial on gaggle.com which I think was helpful but my problem is a bit different.

I am trying to predict diabetes risk based upon Age, Sex...and other factors given this data: http://www.healthindicators.gov/Indicators/Diabetes-new-cases-per-1000_555/Profile/ClassicData

The data gives new cases people per 1,000 people for each dimension (Age, Sex...etc). What I would like to do is devise a way to predict, given a list of dimensions (Age, Sex...etc) a probability factor for a new diagnosis.

So far my strategy is to load this data into R and use some package to create a decision tree, similar to what I saw in the titanic example on kaggle.com, then feed in a dimension list. However, I am a bit overwhelmed. Any direction on what I should be studying, packages/methods/examples would be helpful.

On the bright side, though, using aggregate data like this is fairly straightforward, but requires some probability theory. If $D$ is whether the person has diabetes and $F_1,\ldots,F_n$ are the factors from that link you provided, and if I'm doing my math correctly, we can use the formula: $$\text{Prob}(D\ |\ F_1,\ldots,F_n) \propto \frac{\prod_{k=1}^n \text{Prob}(D\ |\ F_k)}{\text{Prob}(D)^{n-1}}$$ (The proof for this is an extension of the one found here). This assumes that the factors $F_1,\ldots,F_n$ are conditionally independent given $D$, though that's usually reasonable. To calculate the probabilities, compute the outputs for $D=\text{Diabetes}$ and $\neg D=\text{No diabetes}$ and divide them both by their sum so that they add to 1.
Suppose we had a married, 48-year-old male. Looking at the 2010-2012 data, 0.73% of all people get diabetes ($\text{Prob}(D) = 0.73\%$), 0.77% of married people get diabetes (\text{Prob}(D\ |\ F_1)$$= 0.77\%), 1.02% of people age 45-54 get diabetes (\text{Prob}(D\ |\ F_2) = 1.02\%), and 0.70% of males get diabetes (\text{Prob}(D\ |\ F_3) = 0.70\%). This gives us the unnormalized probabilities:$$ \begin{align*} P(D\ |\ F_1,F_2,F_3) &= \frac{(0.77\%)(1.02\%)(0.70\%)}{(0.73\%)^2} &= 0.0103 \\ P(\neg D\ |\ F_1,F_2,F_3) &= \frac{(99.23\%)(98.98\%)(99.30\%)}{(99.27\%)^2} &= 0.9897 \end{align*}$$After normalizing these to add to one (which they already do in this case), we get a 1.03% chance of this person getting diabetes, and a 98.97% chance for them not getting diabetes. • Thank you! So if I have this right, my probability for a white male, age 45-64, for 2010-2012 would be$$ P = {7.0 * 12.0 * 6.7 \over 7.3^2}/1000 = 0.01056108087 $$Meaning there is about a 1.056% chance of diagnosis for an individual with that profile. – terrigenus Aug 11 '15 at 23:47 • Almost. You forgot the normalization step. You need to also compute the value for not getting diabetes:$$ \frac{993.0 * 988.0 * 993.3}{992.7^2}/1000 = 0.98889591936 $$And then normalize so that these two numbers add to one (as probabilities should):$$ P = \frac{0.01056108087}{0.01056108087 + 0.9888959194} = 0.01056681865\$ For a 1.057% chance. With these particular sets of numbers, the normalization step doesn't make much of a difference. With other numbers, though, leaving off the normalization step can lead to rather nonsensical results. – Joshua Little Aug 12 '15 at 0:33