Bit of a noob in this stats world, so apologies in advance for any naiveté. I did a fair bit of stats long ago in college but it's a distant memory, so please assume little knowledge!

The dataset I've got is of the amount of money spent by medical practices in a given month. There already exists a formula that weighs the population (produced long ago using different data) to account for ages (old patients cost more, 20-somethings less etc.), however there are lots of other factors, that we have data for, it isn't weighing. I'd like to try and do that.

Provisional regressions demonstrate a reasonable correlation between, for example, '% of patients with a long term condition' and the existing weighted spend per head of population. Correlation between weighted spend and % long term condition

So my question is: what approach should I take to improve the population weighting to account for these other factors, for which there is a correlation.

Ideally I'd like to generate a formula such as:

Improved weighted population = Existing weighted population * (x * %long term condition) * (y * population deprivation levels)...

The aim would be to improve the error between the current prediction (spend__per_astroPU) and the observed true result (spend_per_raw_head).

I've put a sample of the dataset below, and a link to a python notebook with the full dataset imported into dataframes.

drug_spend  Total Population    astroPU_weight  ASTROPU2013_Population  spend_per_raw_head  spend__per_astroPU  % with a long-standing health condition
0   804443.050      4150    1.146   4757    193.842 169.107 58.230
1   17534209.330    19886   1.130   22478   881.736 780.061 60.218
2   3593560.340     9471    1.033   9786    379.428 367.214 61.756
3   3043412.970     7929    1.272   10089   383.833 301.657 57.046
4   11163851.800    13733   1.033   14189   812.922 786.796 54.217

Python Notebook

  • $\begingroup$ Can you give a reference to the "existing formula for weighting" that you mention? It might be easier to respond for those of us not familiar with that literature. $\endgroup$ – shadowtalker Oct 4 '18 at 12:55
  • $\begingroup$ Sure, it's scholar.google.com/… $\endgroup$ – DrMikey Oct 4 '18 at 14:04
  • $\begingroup$ If you click the NIH link on the right you can get the full pdf article $\endgroup$ – DrMikey Oct 4 '18 at 14:04
  • $\begingroup$ Thanks, I won't be able to take a look at this right away unfortunately. If you aren't getting any answers here, you can try to ask a moderator to move it to stats.stackexchange.com, where I think it might be a better fit. $\endgroup$ – shadowtalker Oct 4 '18 at 14:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.