# Tagging documents for doc2vec

I am working on resume parsing script. I am trying to tag documents sentences with TaggedDocument function, provided by gensim.

What I have managed for now is to divide every text into sentence, put into one flat array and give every sentence an i (its order, basically) tag.

tagged_data = [TaggedDocument(words=word_tokenize(_d.lower()), tags=[str(i)]) for i, _d in enumerate(texts_flat)]


For the reason of possible improvement I want to tag every sentence not only with its order but with the name/order of text it is from. For that, I have made a list of lists, where every text is a list and every text contain list of sentences. i.e

texts = [text1 = [sent1, sent2, ...], text2, text3 ...]

1. How to iterate over this kind of document? I came up with smth like  tagged_data = [TaggedDocument(words=word_tokenize(_d.lower()), tags=[str(i) + '.'+ str(j)]) for i, j, _d in enumerate(texts)]  but i get ValueError

ValueError: not enough values to unpack (expected 3, got 2)

1. Is it even going to do anything?

I've managed to write a function that iterates over it for me:

class TaggedDocumentIterator(object):
def __init__(self, doc_list, labels_list):
self.labels_list = labels_list
self.doc_list = doc_list
def __iter__(self):
for idx, doc in enumerate(self.doc_list):
for idy in enumerate(doc):
yield TaggedDocument(words=idy[1].split(), tags=[str(self.labels_list[idx]) + '.' + str(doc.index(idy[1]))])


 docLabels = list(range(len(texts))) data = list(texts) sentences = TaggedDocumentIterator(data, docLabels) 

• out of curiosity (if you're still around), how did this model tagging system go?
– frei
Feb 18 '19 at 8:06
• @frei still working on this project. Still using this function :) This model was pretty good for tagging the resume into blocks, containing some specific info: education, experience, skills etc. Now I moved a bit further, and for each of these blocks I have a model for classification. So, for education block, there are University class, period of education class, specialization class etc. Feb 18 '19 at 12:09