2
$\begingroup$

Currently I'm trying to build a neural network that is able to classify different types of bottles on an image solely based on the shape. The bottles have no label and at first I only used beer and wine bottles. My goal is to compare a neural network that uses feature extraction to one that uses the raw images.

Example input image:

https://i.gyazo.com/c22151a4bee6146cc689fcb8358483dd.jpg

Using openCV to process the images, I tried to obtain only the edges of the bottles and classify the types based on the shape. Currently, I am receiving the following result for the images:

https://i.gyazo.com/8554b06ebf6e8bd520d11222f70c77c5.png

In the next step, I cropped out each bottle in the image and split them into a 2d array of tiles (normally I try with 30x50 but in this image I show it with 10x20)

https://i.gyazo.com/28fb1bc0fa8d61b185ed2245db222cc8.png

For every tile I execute PCA to extract the eigenvectors. In the end I receive a matrix of 10x20 tiles, where each of them contains the direction of the line. As you can see, the (0.0, 1.0) tuples are the vertical lines, and the (1.0, 0.0) tuples are the horizontal lines.

https://i.gyazo.com/bcd291535ea01b6392ab6aef6747de9d.png

Sorry for the long post, here is my question:

Is it possible to use this data do classify between wine and beer bottles? The shape of the neck of the bottle is different but is it representative for building a neural network? I tried to train a model by calculating the gradient (arctan from the tuple) but i didn't manage to get some successful results. Does anyone have a idea how to pre-process the data for a neural network?

Is it better to use the cropped out images of the bottles as image data for the neural network?

I am thankful for any help as this is my first machine learning project and I would really like to get it working.

$\endgroup$
1
$\begingroup$

Well, you are trying so describe the image by predominant line directions and that really sounds fair but a bit of too much work. If you want to you can try a bit of data augmentation to increase your numbers of samples (Mainly affine transformations)

Other possible useful feature would be descriptors such as:

  • SIFT
  • SURF
  • FAST
  • BRIEF

You could try to describe those shapes using other geometrical functions such as skewness (fill the inside of the shape, consider the eigenvalues to define the major axis directions on PCA)

Finally, bottles in the sample image look similar in shape and differ mainly in size, if your model uses resize it might become scale invariant and then your method will decrease too much of your shape information and give bad results.

------------------------------------------------------------------------------------

Answering the comment bellow:

This is why i tried to do it with my approach of cropping out each bottle and split them into a 2d array of tiles (e.g. 50x20)

This inserts a scale invariance to your model, that is good for processes where image-size vary either because of distance or camera resolution but: This throws alway scale information, that might not be helpful when the main discriminant on your image is scale-related.

$\endgroup$
  • $\begingroup$ Thanks a lot for all these great and helpful tips. I will read myself into these topics and try it out. With my approach the sizes actually don't matter as i split each image into a 2d tile array of the same size. $\endgroup$ – Equintox Apr 5 at 13:34
  • $\begingroup$ That is a problem, since size is relevant part of this classification problem. You have to use really detailed shape information $\endgroup$ – Pedro Henrique Monforte Apr 5 at 13:38
  • $\begingroup$ This is why i tried to do it with my approach of cropping out each bottle and split them into a 2d array of tiles (e.g. 50x20). For every Tile i execute a PCA which gives me the direction of the line of the tile. So in the end it doesnt matter if it is a big or small bottle or if my input img is 5000px or only 400px as the output of any is a 50x20 array of vectors $\endgroup$ – Equintox Apr 5 at 16:34
  • $\begingroup$ Check this video: youtube.com/… $\endgroup$ – Pedro Henrique Monforte Apr 5 at 16:59
  • $\begingroup$ thanks i will check it out :) $\endgroup$ – Equintox Apr 6 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.