How to calculate LFW accuracy of face recognition model?

Question

In my research I have observed many of the face recognition algorithms propose their model accuracy interms of LFW dataset accuracy. I understood that LFW is a open source database and I did download that.

I have developed my own DNN model implemented for face recognition which is similar to facenet architecture. Can anyone help with the procedure to calculate LFW accuracy of my model?

Answer 1

FaceNet should embed pictures in a high dimensional space in such a way that similar pictures are close (wrt to a given distance metric) to each other and different images are farther apart.

That means, to calculate the accuracy of your model you need to find your threshold distance d which delivers the highest accuracy. Given such a distance, you can calculate the accuracy like you would do for a binary classifier. That is, the accuracy is the "Number of correctly classified pairs/Number of pairs"

For example, consider the three images A1,A2 of person A and B1 of person B. And say your DNN embedds the images such that the pair-wise L2 distance (i.e., the euclidean distance) is 1,3, and 7. Then you can see that if use a distance threshold of 2 you get 100% accuracy, and otherwise only 66%.

+----------+--------------+-------------+------+------+
|   Pair   | Ground Truth | L2-distance | d=2  | d=4  |
+----------+--------------+-------------+------+------+
| A1 A2    |            1 |           1 |    1 |    1 |
| A1 B1    |            0 |           3 |    0 |    1 |
| A2 B1    |            0 |           7 |    0 |    0 |
+----------+--------------+-------------+------+------+
| Accuracy |              |             | 1.00 | 0.66 |
+----------+--------------+-------------+------+------+

If you are using Tensorflow, you can follow David Sandberg's instructions to caculate evaluation metrics (including Accuracy) of your model here.

IMPORTANT: To compare the performance of your approach to other approaches, you should follow one of the protocols of the LFW dataset. The protocols are described in detail here (see Section 4). Basically, they have a special view on the dataset, which asks you to report the mean accuracy + error rates on 10 folds of that dataset.