3
$\begingroup$

I am trying to build a LSTM model in keras where I have one question with 10 answers but only ONE among them is correct. So basically im tring to build a 10 class classification problem. As most of the research papers are using Mean average precision(MAP) and Mean reciprocal rank(MRR) to evaluate this type of problems, i want to use them as custom metrics.

So my first question: Is the below code correct to calculate mean average precision?

def MAP(ground_label, predict_label):
mAp = 0
map_idx = 0
extracted = {}

for idx_, glab in enumerate(ground_label):
    if ground_label[idx_] != 0:
        extracted[idx_] = 1

key = np.argsort(predict_label)[::-1]
for i, idx_ in enumerate(key):
    if tuple(idx_.tolist()) in extracted:
        map_idx += 1
        mAp += map_idx / (i + 1)

assert (map_idx != 0)
mAp = mAp / map_idx
return mAp

And the second question is how can i modify this snippet of code to use this as a custom metric function because keras custom metric assumes that the inputs are tensors to the function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.