# What can I use to post process an NLP tree generated from the python library spaCy?

Using spaCy as the NLP engine for a chatbot, I call nlp("Where are the apples?").print_tree() and receive:

[{'word': 'are',
'lemma': 'be',
'NE': '',
'POS_fine': 'VBP',
'POS_coarse': 'VERB',
'arc': 'ROOT',
'modifiers': [{'word': 'Where',
'lemma': 'where',
'NE': '',
'POS_fine': 'WRB',
'modifiers': []},
{'word': 'apples',
'lemma': 'apple',
'NE': '',
'POS_fine': 'NNS',
'POS_coarse': 'NOUN',
'arc': 'nsubj',
'modifiers': [{'word': 'the',
'lemma': 'the',
'NE': '',
'POS_fine': 'DT',
'POS_coarse': 'DET',
'arc': 'det',
'modifiers': []}]},
{'word': '?',
'lemma': '?',
'NE': '',
'POS_fine': '.',
'POS_coarse': 'PUNCT',
'arc': 'punct',
'modifiers': []}]}]


I can easily enough parse out (arc, lemma) pairs for where (advmod, where) and (apple, nsubj), and call a function where(apple).

However, this is a really naive way of handling the parsed tree. Any suggestions for how to handle processing this tree? I don't think something like a multilevel SVM would work. Maybe a NN of some kind?

What do you want to do with the chat bot? How you parse it will depend on the final use case and, believe it or not, many people get the job done by simply collecting the POS they want and using some filtering.

If you want to try to maintain more of the data and perhaps abstract it makes sense to try clustering of some kind, perhaps using hierarchical methods, such as the (relateively new) hdbscan. The features on which you cluster will again depend on what you want to achieve.

If you haven't already, check out the spaCy examples for some inspiration!

Once you have a corpus with word all tagged, you can try training models that might be able to answer questions, based on user input. This will involve steps such as creating encoding of the words (or entire user questions), using embeddings such as Word2Vec, GLoVe, or simple sparse one-hot encodings. You basically need to transform words into numerical input somehow.