I am working on an analysis on proposal rejections across two different periods in time and I was wondering what reflections I would have to make about the risk of bias.

The reason for this is because my first dataset spans across the years 2014 and 2017 where I have: a total of 48 submissions and 19 rejections

my second data set spans over 2018 and 2019 where I have: a total of 11 submissions and 6 rejections. I have made a variety of significance tests that all show that there are not significant differences (chi2 tests) between the proportion of rejections between the two datasets (P>0,05). However, I feel that the still big difference in submissions across the two dataset and their criteria for collection (which years) must mean that I should make some other reflections regarding bias.

Other types of bias:

Selection or sampling bias - I don't think this bias type is relevant, since I am collection all made submissions and rejections.

Also, confirmation bias, I feel, is not really an issue, as I can't really interpret my results any differently than they are, and I have a limited opportunity to try to influence my results in a particular way. A rejection is a rejection.

Over- or underfitting might be an issue in regard to giving a simplistic picture of reality - since I am basing my findings on two quite different scopes in time and that is reflected in the total amount of submission - however, I am unsure if I am understanding this type of bias correctly.

So - are there any other reflections that I could make regarding bias? Or any other types of tests I could make to strengthen my findings?

I appreciate any comments that might help


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.