I am using Spacy for text tokenisation and getting stuck with it:
import spacy nlp = spacy.load("en_core_web_sm") mytext = "This is some sentence that spacy will not appreciate" doc = nlp(mytext) for token in doc: print(token.text, token.lemma_, token.pos_, token.tag_, token.dep_, token.shape_, token.is_alpha, token.is_stop)
returns something that seems to me to say that tokenisation was succesful:
This this DET DT nsubj Xxxx True False is be VERB VBZ ROOT xx True True some some DET DT det xxxx True True sentence sentence NOUN NN attr xxxx True False that that ADP IN mark xxxx True True spacy spacy NOUN NN nsubj xxxx True False will will VERB MD aux xxxx True True not not ADV RB neg xxx True True appreciate appreciate VERB VB ccomp xxxx True False
but on the other hand
[token.text for token in doc.lefts]
returns an empty list. Is there a bug in lefts/rights?
Beginner at natural language processing, hope I am not falling into a conceptual trap. Using Spacy v'2.0.4'.