The problem is the following:

I have multiple datasets for which I want to calculate a ranking for each. All observations contained in the datasets can be arbitrarily permuted, so they are unpaired, to speak in the words of statisticians.

Example datasets are:

dataset1 = [0.6487500071525574, 0.6499999761581421, 0.6412500143051147, 0.6662499904632568, 0.6225000023841858, 0.6324999928474426, 0.637499988079071, 0.6287500262260437, 0.6412500143051147, 0.6212499737739563]

dataset2 = [0.6075000166893005, 0.6287500262260437, 0.6312500238418579, 0.6162499785423279, 0.6012499928474426, 0.6150000095367432, 0.6387500166893005, 0.6200000047683716, 0.5950000286102295, 0.5849999785423279]
dataset3 =[0.6237499713897705, 0.612500011920929, 0.6075000166893005, 0.6162499785423279,  0.6187499761581421, 0.6287500262260437, 0.6200000047683716, 0.6237499713897705, 0.5824999809265137, 0.5787500143051147]

I understand for datasets with paired observation, one would simply rank each observation column-wise and simply average over the average each observation has in each dataset. Example:

ranks_dataset2 = [3, 2, 2, 2.5, 3, 3, 1, 3, 2, 3]
=> Avg.Rank: 2.45

But how would I do this for unpaired observations?


1 Answer 1


I think I figured out a nice way on how to rank the datasets:

Ranking formula

The rank r(D) of dataset D is calculated by subtracting from the total number of datasets N_D the sum of the number of wins W_i of each observation i in D averaged over the total number of observations |D| in respect to the complete set of observations without |D|, which is N - |D|, multiplied the number of remaining datasets N_D - 1. So if all observations in D have N-|D| wins, the rank of the dataset |D| is 1. If all observations have no wins (they are 0), the rank of the dataset |D| is simply N_D.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.