1
$\begingroup$

I'm currently working on implementing a bag of visual words in Python. I get the general gist of how it works but I can't seem to find any sources that explain it in more detail to a level where I can implement it. I'm guessing scikit learn and scikit image would come in but I can't seem to point myself in the right direction. Any help?

$\endgroup$
  • $\begingroup$ Welcome to the site. Do include what you learnt about the approach, and what is causing you confusion :) $\endgroup$ – Dawny33 Mar 1 '16 at 16:00
  • 1
    $\begingroup$ Thanks, so I basically know that a visual bag words is essentially a "count" of how many of each feature are in an image. So if we had an image of a face the features would be the eyes, the hair, the nose etc. and the BoVW would be how many of each we have. The confusion's coming from how this applies in a CS context when we use more technical features such as colour histograms. $\endgroup$ – user3396592 Mar 1 '16 at 16:08
  • $\begingroup$ This is something I'm interested in learning fairly soon and would attempt to provide an answer if I knew more in detail, but programmingcomputervision.com (there's a free PDF on the site) has a great deal of information about computer vision. There's a small section on how visual words work along with some Python code to test it out. $\endgroup$ – ImperativelyAblative Mar 1 '16 at 17:22
2
$\begingroup$

The necessary information can be found on wikipedia: https://en.wikipedia.org/wiki/Bag-of-words_model_in_computer_vision

"when we use more technical features such as colour histograms" Judging from this sentence I guess you need to understand the "Codebook" generation.

First step is to extract features of patches in the image. For efficiency you only want to take patches which are interesting and calculate discriminative features on them. SIFT is one Method which performs both steps for you. It takes care of finding good spots and it calculates features on this spot.

Now you can generate your codebook. A codebook will map every possible featurevector ( after all they're just numeric vectors ) to a certain output codeword. One possibility to do this, is to use k-means for codebook generation. After you built your codebook, a vector is mapped to a code by finding the minimal distance to all the entries ( since you used k-means you can use euclidean distance ).

Now you have a complete realization of the bag of words model. You can now dive into using it for classification. The required algorithms can be implemented using the libraries you mentioned.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.