I am trying to build a LSTM model in keras where I have one question with 10 answers but only ONE among them is correct. So basically im tring to build a 10 class classification problem. As most of the research papers are using Mean average precision(MAP) and Mean reciprocal rank(MRR) to evaluate this type of problems, i want to use them as custom metrics.

So my first question: Is the below code correct to calculate mean average precision?

def MAP(ground_label, predict_label):
mAp = 0
map_idx = 0
extracted = {}

for idx_, glab in enumerate(ground_label):
    if ground_label[idx_] != 0:
        extracted[idx_] = 1

key = np.argsort(predict_label)[::-1]
for i, idx_ in enumerate(key):
    if tuple(idx_.tolist()) in extracted:
        map_idx += 1
        mAp += map_idx / (i + 1)

assert (map_idx != 0)
mAp = mAp / map_idx
return mAp

And the second question is how can i modify this snippet of code to use this as a custom metric function because keras custom metric assumes that the inputs are tensors to the function.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.