Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
https://en.wikipedia.org/wiki/Postage_stamp
Is it possible through AI,Computer vision & Machine learning technologies by giving details of the Stamp after inputting the Stamps images?
Input : Stamp images in .gif,.jpg,.tiff format.
Output :
Stamp country origin :
Stamp value with currency :
Stamp category :
Stamp description :
Detecting data on a stamp image is no easy tasks, they are designed to be at least complicated to fake and have a large variation. Problems are:
If you have a dataset with all possible stamps, that reduces to a image registration problem which can be solved with classical computer vision (no machine learning). ML can do it too, just using the same architectures used for facial recognition that accepts new registries.
It depends on the stamp. If the stamps contain all the information you say you need then it'll work. But as the stamps shown in your link you will need a large amount of data for your model to generalize well. In my experience stamps convey the information quite differently therefore for your model to do well you will need a complex model from scratch as most pre-trained models haven't been trained on stamps. Therefore new weights will be required.
Your question is not trivial and is actually quite a large project.
I am working on this problem right and now I've discovered a number of problems and solutions to getting this to work for postage stamps. They are unique objects that are difficult to do computer vision on because of their perforated edges. It's both a blessing and a curse:
Catalog your data: You need a catalog of existing stamp data. This includes all of the information you would like, such as country, value, currency, description, and category. Very little of this information can be determined using computer vision. You might be able to get the value and currency from a stamp's front face, but none of the other stuff. Like, how would you determine country? Or a description that include details like year of issue, etc? Honestly if you have all of the other information, just add value and currency to your data set. You then need to pair this info to a stamp's image. In this case, using a MySQL database to pair the stamp row id to an image_id would work. Now you have the data you can match images to.
Train a ML model to find generic stamp objects: Finding a stamp in an image is the first step on the vision side of things. If you know all of your images to be matches are directly of cropped stamps, you can skip this step. But unless your images are perfect (which they never really are), you should train a ML model to find stamps in any image, putting a bounding box around one, and then cropping the image to the box.
Crop images to get only core info: This is where things get tricky. Because of the perforated edges of a stamp, image matching techniques like using SIFT will see features in your edges. These features will match up to almost all of the images in your set, and cause image recognition to not work well. So what I do before running image matching is I'll crop the borders of each stamp out (85% of the original size) to build my image matching index with. Getting rid of common data is key to making each stamp more unique, and perforated edges, or just borders in general, are a common denominator in all stamps that we can try and filter out.
Exact stamp matching with SIFT: After that, I run OpenCV SIFT detection of about 150 features per image, each revised to 300H x AspectW, to start building a descriptors list. I run PCA on the vectors down to 64 dimensions, and then store it all in a Faiss index. There are lots of little details I am skipping over here (and you can look up the advantages and disadvantages of everything I mentioned here), but this is the gist of it.
Further filtering with color and weights: I then run the image through the same process of resizing and feature detection, and then match that against that Faiss index. Right now I'm just taking the top vote getter (based on the number of features detected) as the match, and it works extremely well, but you can add further checks like Color Moments or histogram matches to further weed out mismatches with high vote counts. You can also filter more by adding in weight to scores based on distance between features.
Of course, this is all just to get the feature matching to work. Beyond that you have the scope of whatever application/service you're building around this, which includes UI, camera control, etc etc.
I hope this helps! It's a massive project.
