Using spaCy as the NLP engine for a chatbot, I call nlp("Where are the apples?").print_tree() and receive:

[{'word': 'are',
  'lemma': 'be',
  'NE': '',
  'POS_fine': 'VBP',
  'POS_coarse': 'VERB',
  'arc': 'ROOT',
  'modifiers': [{'word': 'Where',
    'lemma': 'where',
    'NE': '',
    'POS_fine': 'WRB',
    'POS_coarse': 'ADV',
    'arc': 'advmod',
    'modifiers': []},
   {'word': 'apples',
    'lemma': 'apple',
    'NE': '',
    'POS_fine': 'NNS',
    'POS_coarse': 'NOUN',
    'arc': 'nsubj',
    'modifiers': [{'word': 'the',
      'lemma': 'the',
      'NE': '',
      'POS_fine': 'DT',
      'POS_coarse': 'DET',
      'arc': 'det',
      'modifiers': []}]},
   {'word': '?',
    'lemma': '?',
    'NE': '',
    'POS_fine': '.',
    'POS_coarse': 'PUNCT',
    'arc': 'punct',
    'modifiers': []}]}]

I can easily enough parse out (arc, lemma) pairs for where (advmod, where) and (apple, nsubj), and call a function where(apple).

However, this is a really naive way of handling the parsed tree. Any suggestions for how to handle processing this tree? I don't think something like a multilevel SVM would work. Maybe a NN of some kind?


What do you want to do with the chat bot? How you parse it will depend on the final use case and, believe it or not, many people get the job done by simply collecting the POS they want and using some filtering.

If you want to try to maintain more of the data and perhaps abstract it makes sense to try clustering of some kind, perhaps using hierarchical methods, such as the (relateively new) hdbscan. The features on which you cluster will again depend on what you want to achieve.

If you haven't already, check out the spaCy examples for some inspiration!

Once you have a corpus with word all tagged, you can try training models that might be able to answer questions, based on user input. This will involve steps such as creating encoding of the words (or entire user questions), using embeddings such as Word2Vec, GLoVe, or simple sparse one-hot encodings. You basically need to transform words into numerical input somehow.

I hope this gives you some keywords to help you on your search :-)

  • $\begingroup$ Yess, this helps, thank you. And where I am write now is get the POS from spaCy and pulling out (i.e. nsubj and pobj), then mapping those onto bot commands. As for the nature of the bot, it's a champs bot - need to be able to ask where certain services are running, what the state is, how oversubscribed we might be on cloud resources. $\endgroup$ Jul 23 '18 at 22:31
  • $\begingroup$ Actually you know what, this led me towards word vectors and similarity comparisons. Since I have a limited number of sentences (intents, really) that the bot will handle, I can just do spacy similarity comparisons and pull out prepositional objects. Quick prototype is working. Thanks! Thanks! $\endgroup$ Jul 24 '18 at 21:13
  • $\begingroup$ Sheesh. New macbook, just disabled autocorrect on it. So yes thanks for the word2vec suggestion, I'm writing a chatops bot, not a champs bot, and I only meant to thank you once. Cheers. $\endgroup$ Jul 24 '18 at 22:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.