I am working on an analysis on proposal rejections across two different periods in time and I was wondering what reflections I would have to make about the risk of bias.
The reason for this is because my first dataset spans across the years 2014 and 2017 where I have: a total of 48 submissions and 19 rejections
my second data set spans over 2018 and 2019 where I have: a total of 11 submissions and 6 rejections. I have made a variety of significance tests that all show that there are not significant differences (chi2 tests) between the proportion of rejections between the two datasets (P>0,05). However, I feel that the still big difference in submissions across the two dataset and their criteria for collection (which years) must mean that I should make some other reflections regarding bias.
Other types of bias:
Selection or sampling bias - I don't think this bias type is relevant, since I am collection all made submissions and rejections.
Also, confirmation bias, I feel, is not really an issue, as I can't really interpret my results any differently than they are, and I have a limited opportunity to try to influence my results in a particular way. A rejection is a rejection.
Over- or underfitting might be an issue in regard to giving a simplistic picture of reality - since I am basing my findings on two quite different scopes in time and that is reflected in the total amount of submission - however, I am unsure if I am understanding this type of bias correctly.
So - are there any other reflections that I could make regarding bias? Or any other types of tests I could make to strengthen my findings?
I appreciate any comments that might help