I have experimented with a number of different machine learning models used for pose estimation. Most of them output a heatmap and offsets for the detected person(s) in the image. I really like the performance/accuracy of the multipose estimation model here.

What I would like to do next is to create a model similar to this one, except it should label each pose of the person(s) detected. There are multiple different implementations caffe/pytorch/tensorflow to choose from. I've thought about how to approach this and I have thought of a few different ways:

  1. Create a completely new machine learning model and use the labeled output of the pose estimation model to train it.
  2. Change or add layers to the machine learning model to change the output. (Not sure how this is done)
  3. Ditch the pose estimate model and train a new model to directly estimate using raw images/labels of cropped people. This would rely on another method to detect each person.

I want to take the path of least resistance here but I also care about the time it takes to gather/process data, and most importantly the accuracy/performance of the model. Are there any experienced Machine Learning/Data Scientists who answer the following?

  • Which approach should I take? advantages/disadvantages
  • Which machine learning library offers the functions to do this.
  • My assumption is that option 1 or 2 would be more accurate than option 3. Am I correct?

I worked in a similar kind off object detection problem.so, my suggestion is Transfer Learning.

i) The optimal way is to go with transfer learning.It allows us to build accurate models in a timesaving way.With transfer learning, instead of starting the learning process from scratch, you start from patterns that have been learned when solving a different problem. This way you leverage previous learnings and avoid starting from scratch.

A pre-trained model is a model that was trained on a large benchmark dataset to solve a problem similar to the one that we want to solve. Accordingly, due to the computational cost of training such models, it is common practice to import and use models from published literature (e.g. VGG, Inception, MobileNet).

Several pre-trained models used in transfer learning are based on large convolutional neural networks (CNN).

When you’re repurposing a pre-trained model for your own needs, you start by removing the original classifier, then you add a new classifier that fits your purposes, and finally you have to fine-tune your model according to one of three strategies:

1.Train the model.
2.Train some layers and leave the frozen.
3.Freeze the convolutional base.

Approaches for Transfer learning process:

1)Select a pre-trained model.
2)Classify your problem according to the Size-Similarity Matrix. 
3)Fine-tune your model.

ii) Tensorflow offers a vision model named as PoseNet.

PoseNet is a vision model that can be used to estimate the pose of a person in an image or video by estimating where key body joints are. From here you can freeze the model and fine tune it. orelse you can use other pretrained image model (e.g. mobilenet , resnet).

iii) The estimation directly from the raw images first request you to identify or detect the people , so it relays on another model.

  • $\begingroup$ I don't understand what you mean when you say 'fine tune your model.' Can you give an example? Can you provide a reference for 'freezing' and 'fine-tuning' a pretrained model? $\endgroup$ Feb 18 '20 at 2:34
  • 1
    $\begingroup$ learnopencv.com/… ..try this ..has code and explanation .. i would have explained it but can't do it in one comment box :) $\endgroup$ Feb 18 '20 at 5:33
  • 1
    $\begingroup$ sorry, just one other thing ..unless you are super comfy with understanding how each layer helps (typically layers closer to input understand more "meta" information, while the layers closest to output "specialize") i would recommend using the pre trained model to extract features and then passing them into your own dense layers / RNN+dense layers for final classification on your custom data set.. finetuning needs a lot of playing around since there's a whole lot of hyper parameters that could improve / worsen results. But definitely worth understanding , if i haven't confused u already $\endgroup$ Feb 18 '20 at 5:44
  • $\begingroup$ @josh its not easy to provide some examples for fine tuning models, I nearly spent months to find out some parameter and layer optimization and tuning.you can find some basic examples in websites,but research papers talks a lot about this..But the thing is its worth for spending some time on this topic.you will get to know about the real. $\endgroup$ Feb 18 '20 at 10:45
  • $\begingroup$ Thank you @VikramMurthy I will follow the information in that link you shared with me. $\endgroup$ Feb 19 '20 at 0:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.