0
$\begingroup$

I am using doc2vec to vectorize input text. I am converting my input dataset to tagged data and giving it as input.

Initially I tried with a data of 27 input text:

tagged_data = [TaggedDocument(words=word_tokenize(_d.lower()), tags=[str(i)]) for i, _d in enumerate(X)]

max_epochs = 125
vec_size = 20
alpha = 0.05

model = Doc2Vec(size=vec_size,
            alpha=alpha, 
            min_alpha=0.00025,
            min_count=1,
            dm =0)

model.build_vocab(tagged_data)

for epoch in range(max_epochs):
    print('iteration {0}'.format(epoch))
    model.train(tagged_data,
            total_examples=model.corpus_count,
            epochs=model.iter)
    # decrease the learning rate
    model.alpha -= 0.0002
    # fix the learning rate, no decay
    model.min_alpha = model.alpha

model.save("d2v.model")
print("Model Saved")

print ("docvecs")
print(model.docvecs)
print(type(model.docvecs))
print(len(model.docvecs))

I am getting the length of the docvecs as 27 which is correct as I was having 27 input text to the model and tagged 27 unique values.

I later fitted this to a logistic regression model:

train_arrays=np.zeros([27,20])
train_labels=np.zeros(27)
for i in range(0,27):
    #x_train.append(tagged_data[i][0])
    train_arrays[i]=(model.docvecs[i])
    train_labels[i]=i



print ("-------------------------")
print (x_train)
print(y_train)
from sklearn.linear_model import LogisticRegression
logreg = LogisticRegression(n_jobs=1, C=1e5)
logreg.fit(train_arrays, train_labels)
print("logistic regression fiitted properly..")

I tested this with an infinite loop and it works fine:

 while True:
    print("Enter text : ")
    usr=input()

    usr=word_tokenize(usr.lower())
    v1 = model.infer_vector(usr)

    print("vector of input text.. ",v1)

    sims = model.docvecs.most_similar([v1])
    print(sims)
    test_array=np.zeros([1,20])
    test_array[0]=v1
    ans=logreg.predict(test_array)
    print(ans)

I am getting the output correct:

Enter text :
what will my business card carry
vector of input text..  [-0.36948422 -0.3151284  -0.392992   -0.56482047  0.17411898  0.16804925
  0.3298428  -0.3225111   0.06729688 -0.02223648 -0.07785773 -0.15621385
  0.34434605  0.30244747  0.08436651 -0.2911789  -0.02142929  0.14122409
 -0.4378101   0.32535276]
[('14', 0.9742787480354309), ('4', 0.7727420330047607), ('6', 0.7526462078094482), ('7', 0.7040897011756897), ('21', 0.6696580648422241), ('19', 0.6558915376663208), ('9', 0.6502488851547241), ('22', 0.6467552185058594), ('15', 0.6378635168075562), ('13', 0.6291216611862183)]
[14.]

Now, the problem is that I am using a different data set of 298 input text and there are 29 intent_id's for these 29 groups. So of the 298, the text data has an average of 7-8 in each group.

So when preparing tagged data, I used the intent_id to tag and feed to doc2vec. Groups are from 1-29:

TaggedData=[]
for i in range(0,len(train_data)):
    print(i,train_data['utterance'][i],train_data['intent_id'][i])
    tagDoc=TaggedDocument(words=word_tokenize(train_data['utterance'][i].lower()),tags=[train_data['intent_id'][i]])
    TaggedData.append(tagDoc)

When I feed this to the doc2vec model and take the length of len(model.docvecs) it is coming as 30. I have used the same code above. Ideally, this should come as 29 (what I was expecting). I would like to feed the vector output of 298 input text to logistic regression. However, here the length is only 30 which I am not able to understand. What is the correct thing to do here?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.