Bag of Words creation in image

Question

In SIFT feature extraction how the key points will be generated and how the features will be stored in the database. In image will the bag of visual words be images or text words?

Answer 1

How to calculate key points:

1. Take your image $I(x,y)$ and convolve it using a Gaussian $G(x,y,k\sigma$) at different scales $k \sigma$ to obtain $L(x,y,k\sigma)$. This produces several versions (one for each scale $k\sigma$) of your image $I(x,y)$ with different degrees of blurring.

2. Separate your blurred images according to octaves (an octave is usually taken as $2\sigma$). Within a given octave, take the difference between adjacent blurred images $L(x,y,k_{i}\sigma)$ and $L(x,y,k_{j}\sigma)$. This difference is called a Difference of Gaussians (DoG) $D(x,y,\sigma) = L(x,y,k_{i}\sigma) - L(x,y,k_{j}\sigma)$. At this point, an image will be helpful (source: opencv):

As you can see, you have the blurred/convolved/filtered images on the left and the differences between adjacent images on the right.

3. Then, take one pixel on a DoG and compare it to its 8 neighbors in the same DoG, as well as to the 9 equivalent pixels of the DoG located in the next scale and to the other 9 pixels in the previous DoG. In the drawing, this corresponds to the DoG's located above and below. If this pixel is a local extrema, then you have a candidate keypoint. Keep in mind that candidates can be discarded based on other criteria.

Once you have these keypoints, you proceed to generate descriptors. For each keypoint, calculate an image gradient. As you should know, gradients tell you the direction of maximum rate of change. Therefore, you can construct a grid around each keypoint that is oriented according to the dominant gradient around that point. This grid has subregions (usually 16 subregions) and for each subregion, calculate an 8-bin histogram. Finally, concatenate every histogram you obtained from every subregion in your grid and that is the feature vector for that keypoint (actually, the full feature vector also includes the location and rotation angle.) A helpful illustration (from Solem's book Programming Computer Vision with Python):

At this point, you have a feature vector with 132 values for each keypoint. As for your second question, I'm not sure about the best approach to store an array in a database. Maybe others can expand on this point. Of course, there are several options:

1. Create an array datatype to store your data
2. Simply use a VARCHAR field
3. Store it as a binary file
4. Use a database specifically designed to handle arrays.