I am trying to train a neural network to detect objects within a tattoo. I couldn't find any existing labeled dataset so I need to manually create and label my own. I only understand the basics of machine learning and was going to start by using TensorFlow's Object Detection (https://github.com/tensorflow/models/tree/master/research/object_detection)

For my labeling I am stuck between two options. Using https://scontent-ort2-1.cdninstagram.com/vp/85794872ffd8b8166b1dd559fcf718b0/5C90FA77/t51.2885-15/e15/11326797_987161747982140_1156040219_n.jpg as an example:

Approach 1: Create a simple bounding box around each object in the tattoo: Multiple boxes: 1 around the rose, the clock and the words. The box for the clock however may bleed into the roses, if the tattoo was more complicated and compact it would bleed a lot (Bleed as in parts of the rose are in the clock's bounding box).

Approach 2: Cut out each object exactly so there is no bleeding and every object is exactly what it is and fill the negative space with black.

I understand that background noise is important. When the NN is training against many images it detects that the background isn't significant and can pick up the features of just object being detected, however, since all my images are tattoos which are on people's bodies the background for everything is just skin tone. I am afraid if I don't have an exact depiction of the tattoo object it will not train accurately especially since every tattoo is different.

I am not sure which approach to take or if I should go a different directions. Thanks

  • $\begingroup$ You need to do "Object detection" not "image classification".You need a data set of images with bounding boxes and labels in meta data. With ML you want your data set for training as close to the data that will be used with the model. Therefore I do not think you want to crop parts and blank out the rest. Simplistically it need to know what is not what you are looking for as much as what you are looking for. $\endgroup$ – William J Bagshaw Jan 18 '19 at 8:56
  • $\begingroup$ I would like to do something else with tattoos, tattoo recognition, ie. recognise that 2 different pictures of the same tattoo are the same tattoo. Do you have a dataset already and is it open source? If so, it would be really helpful if you could share it :) $\endgroup$ – scc May 29 '19 at 11:57

It is really hard to give an answer on what would work best, as Deep Learning is often based on experimenting with different things. But I would highly recommend you start quick and dirty. This is an advice given at all universities or by the famous Andrew Ng. Just start with the easiest model and then iterate. For example if your simple model doesn't work try to create a more complex pipeline by preprocessing the images as you suggested.

In Deep Learning, time is most often the biggest factor. Don't waste it by trying to create a highly complex model to then just observe that it didn't help to improve your model.

I would like to add, if you decide to add more parts to your pipeline, for example 1) Find the human in the image 2) Find the tattoo 3) Classify tattoo etc. you should take a look at this video as it explains how to improve your pipeline in the most efficient way.


Do not do 2). You need to do "object detection" using a data set of images with labeled bound boxes as metadata. It needs to know what you are not interested in so blanking out the rest of the image will not help.

If you blank out bits you will create a tattoo detector, not what you want.

You can greatly increase your dataset by distorting the training data such as blur, noise, flip rotate crop etc. You will need to change the metadata too. Your training workflow may allow you to automatically do this.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.