1
$\begingroup$

My goal is to train a neural network to recognize faces based on a list of landmark points generated by Google Firebase ML-Kit. Since I just started familiarizing myself with ML, I only want my model to recognize if a face belongs to a person in a smaller set of people and if so, to which one. Hence, each face is represented by a list of (x, y) coordinates denoting each landmark point recognized by the Google library. Variations in the absolute locations of the points and their relative position to each other are cues telling people apart.

Therefore, from a list of points such as this one:

Facial landmarks

I get a list of coordinates with the person one-hot encoded at the end:

1975.72,120.265564,2108.6755,141.79669,2344.71,214.35474,[...], 000010000

and I want to create a .csv file containing multiple rows for each person.

My problem is, that I do not know how to represent that the points are actually grouped by two as (x, y) coordinate of the very same point in a way that my model can understand it.

Since I'm new to this, I'm not even sure if my model actually needs to understand this in order to recognize someone out of a list of points so sorry, if this is a stupid question. Also, I do not need the problem to be solved, I only request help with the notation system in my training/test/validation data.

Clarification

I already know how to extract the list of points from an image using Google ML-Kit. What I do not know is how to represent points in a way that the net will pick up that x and y belong together constructing a 2d object.

For example, I could have a raw csv file like in the example and all I would need to do is splice the content alongside the commas, but then the net would never know about the relationship. Instead of a list of 1d values, however, I could also expose the net to a list of 2d points such as [[x, y], [x, y], [.....]], but I don't know if data with n-dimensional elements needs any special attention or the net can understand it as if it were a regular list with consecutive elements.

$\endgroup$
1
$\begingroup$

You are describing is a Cartesian coordinate system where each point is an ordered pair of signed numbers.

In order to compare across people, you need to define the same origin (0,0) and define all points relative to that. Typically, the origin of a face is the tip of the nose.

The input to a model would be a collection of these Cartesian coordinate pairs for each training instance.

Google's Firebase ML-Kit has a very specific API for Face Detection with specific facial landmarks and each point is a VisionPoint. Given the predefined, rigid, and hierarchical structure, it would be more useful to store the data as JSON to preserve the relationship between label and coordinate pair.

$\endgroup$
1
  • $\begingroup$ Thank you for your answer. So this means that I can easily use a collection of n dimensional objects as training data, right? Does this need some special manipulations? $\endgroup$ – rTECH Mar 7 '20 at 19:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.