2
$\begingroup$

I'm working on an animal classification problem, with the data extracted from a video feed. The recording was made in a pen, so the problem is quite challenging with a dark background and many shadows: enter image description here

Initially I tried scikit-image, but then someone helped me with an advanced tool called crf-rnn (http://crfasrnn.torr.vision/) that does a great job segmenting and labelling objects in an image. I did the following: 

import caffe
net = caffe.Segmenter(MODEL_FILE, PRETRAINED)
IMAGE_FILE = '0045_crop2.png'
input_image = caffe.io.load_image(IMAGE_FILE)
from PIL import Image as PILImage
image = PILImage.fromarray(np.uint8(input_image))
image = np.array(image)
mean_vec = [np.mean(image[:,:,vals]) for vals in range(image.shape[2])]
im = image[:, :, ::-1]
im = im - reshaped_mean_vec
cur_h, cur_w, cur_c = im.shape
pad_h = 750 - cur_h
pad_w = 750 - cur_w
print(pad_h, pad_w, "999")
im = np.pad(im, pad_width=((0, max(pad_h,0)), (0, max(pad_w,0)), (0, 0)), mode = 'constant', constant_values = 255)
segmentation = net.predict([im])
segmentation2 = segmentation[0:cur_h, 0:cur_w]

The resulting image segmentation is rather poor (although two cows are recognized correctly): enter image description here

I use a trained crf-rnn (MODEL_FILE, PRETRAINED), which works well for other problems, but this one is harder. I would appreciate any suggestions on how to pre-process this sort of image to extract the shape of most cows.

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
1
$\begingroup$

It would be appreciated if you could explain precisely what your goal is:

  1. you want to identify what animal is in your picture ?
  2. you want to count the number of animals ?
  3. you want to get the position of each animal in the picture ?

In any case, I know that you can get some already trained neural nets from google or anywhere else. This neural net can be used with caffe as it is the case in this google deepdream stuff on github (look at ):

https://github.com/google/deepdream/blob/master/dream.ipynb

Then, if you want to highlight or identify the positions of your animals, you'll find this article inspiring:

http://www.matthewzeiler.com/pubs/cvpr2010/cvpr2010.pdf

It explain how to reverse convolutional networks to identify what part of your image helped to recognize what is inside. The found projection get you something similar to your second picture (called a mask), but depending on the neural net you use, you can get better results.

Improve this answer
$\endgroup$
10
  • $\begingroup$ 'what your goal is': point 2+3; he problem is that the background in hard to extract, as there are a lot of shadows and many cows are of a similar color $\endgroup$
    – Alex
    Mar 22, 2016 at 10:42
  • $\begingroup$ Well, your image still have some good edges. You can try using some edge detection algorithm to accentuate the objects limits: en.wikipedia.org/wiki/Edge_detection. But once again, I think the current network your working with is just not good enough for this case. Did you try some basic preprocessing like contrast or spectral regularization ? $\endgroup$
    – Robin
    Mar 22, 2016 at 11:56
  • $\begingroup$ Do you mean like this: scikit-image.org/docs/dev/api/… $\endgroup$
    – Alex
    Mar 22, 2016 at 14:44
  • $\begingroup$ I think edge-based algorithms are more adapted since neural networks and image recognition are mostly based on edges (and not colors). So you should try amplifying the edges on your image. $\endgroup$
    – Robin
    Mar 22, 2016 at 20:39
  • $\begingroup$ I've had a bit of success with histogram equalization to increase contrast, but otherwise not sure how to continue. What can I do to maximize the effect of a segmentation algorithm? $\endgroup$
    – Alex
    Mar 22, 2016 at 21:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.