I have a list of items (size ~50K) and several documents( average page per document ~10). I am trying to find what all items are listed in each document as follows :
for document in documents_folder: document_text = extract_content(document) for item in item_list: if item is found in document_text: save(doument,item)
This takes ~5 sec per document to execute.
How can this be optimized ? Is this the best approach ? Is there a better way to do this ?
Any suggestion is appreciated