11
$\begingroup$

Amazon's Alexa, Nuance's Mix and Facebook's Wit.ai all use a similar system to specify how to convert a text command into an intent - i.e. something a computer would understand. I'm not sure what the "official" name for this is but I call it "intent recognition". Basically a way to go from "please set my lights to 50% brightness" to lights.setBrightness(0.50).

The way they are specified is by having the developer provide a list of "sample utterances" which are associated with an intent, and optionally tagged with locations of "entities" (basically parameters). Here's an example from Wit.ai:

wit.ai example

My question is: how do these systems work? Since they are all very similar I assume there is some seminal work that they all use. Does anyone know what it is?

Interestingly Houndify uses a different system that is more like regexes: ["please"] . ("activate" | "enable" | "switch on" | "turn on") . [("the" | "my")] . ("lights" | "lighting") . ["please"]. I assume that is integrated into the beam search of their voice recognition system, whereas Alexa, Wit.ai and Mix seem to have separate Speech->Text and Text->Intent systems.

Edit: I found a starting point - A Mechanism for Human - Robot Interaction through Informal Voice Commands. It uses something called Latent Semantic Analysis to compare utterances. I'm going to read up on that. At least it has given me a starting point in the citation network.

Edit 2: LSA is essentially comparing the words used (Bag of Words) in each paragraph of text. I don't see how it can work very well for this case as it totally loses the word order. Although maybe word order doesn't matter much for these kinds of commands.

Edit 3: Hidden Topic Markov Models look like they might be interesting.

Improve this question
$\endgroup$
6
  • $\begingroup$ This post explains intent classification in details: mrbot.ai/blog/natural-language-processing/… $\endgroup$
    – znat
    Nov 28, 2017 at 21:33
  • $\begingroup$ This appears to use the "bag of words" method I mentioned in my question. Basically just add up the word vectors in the sentence. That can't be how it works though. Wit and Nuance's interfaces show that they recognise entities which bag of words can't easily do. Also bag of words loses all ordering so something like "Set an alarm for 10 past 5" would be indistinguishable from "Set an alarm for 5 past 10". There must be something more going on. $\endgroup$
    – Timmmm
    Nov 29, 2017 at 11:40
  • $\begingroup$ Entity extraction is another problem where sequence matters. If you have a lot of data an Rnn will work, in smaller datasets, which are frequent in chatbots, conditional random fields work very well $\endgroup$
    – znat
    Nov 29, 2017 at 15:30
  • $\begingroup$ Ok, so ... I'm looking for a more detailed answer than "an RNN will work". Most modern sequence learning systems use RNNs so that seems a given. $\endgroup$
    – Timmmm
    Nov 29, 2017 at 18:15
  • $\begingroup$ intents are about the general meaning of the sentences (avg of vectors) and entities are about learning the context (surrounding words) in which they appear. Rnn or CRF are just algorithms that can be used because they learn from sequences. If you want to learn in detail, look into Rasa source code $\endgroup$
    – znat
    Nov 29, 2017 at 18:24

2 Answers 2

Reset to default
4
$\begingroup$

Although not directly answering your question, you may be interested in the field of automated question answering. To answer natural language text questions they must first be understood, which overlaps with your problem.

A good resource is the course by Jurafsky and Manning. Particularly the sections on semantics and question answering may help with what you are looking for. There are accompanying lecture videos available on youtube here.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I find first part of your answer very funny yet informative. $\endgroup$
    – Diego
    Nov 5, 2016 at 0:01
  • $\begingroup$ Perhaps this would be better as a comment since, as you admit, it doesn’t answer the question. $\endgroup$
    – kbrose
    Nov 29, 2017 at 0:04
2
$\begingroup$

This post has an approach. Basically they use bag of words - they convert the words to sparse vectors and then add them up.

It appears to work fairly well but one major flaw is the answer is independent of word order, so you can't do queries like "How many kilos in a pound" unless you special case them.

However I did text with Alexa and it is fairly insensitive to word order changes so maybe they do use something similar.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Curious - what advantage do sparse-vectors have over Naive Bayesian? Both to me solve linearly-separable problems with the naive bag-of-words assumption $\endgroup$
    – Angad
    Oct 25, 2017 at 5:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.