Question and Answer Chatbot for Customer Support

Question

I want to build a chatbot that serves as a first line customer support on a retail website. I have a large log of chat sessions between customers and support professionals that I can use. I am wondering what is the best way to go about building this chatbot.

Here are some ideas I have looked into;

• The first thing I did was to breakdown the chat logs into Q&A pairs. Treat each question as a document, compute a term-document matrix and use it to retrieve the most similar question when a customer asks sometihng. The idea was to then simply pick the answer given to the question by the human (with some modifications). However, this gives really abysmal results and does not match to previous questions very well. Even if I get this approach to work well, we would be limited to the questions that already been asked.
• Another thing I thought about was to modify something like ALICE and write some AIML for customer support related QA. However, that would take a lot of very precise AIML writint to work well. This solution would not be able to scale to other languages, which is something I want to do.
• Another idea I have is to try and cluster the answers given by the customer support staff (they are more likely to be aligned compared to the questions asked by customers). This would give me a sense of what questions are similar. Then I can use something like LSA on the questions and fall back to the first approach.
• Yet another way is to build an ontology of the domain specific knowledge and given a question decide which category the question falls into. The questions from the chat logs can then be mapped to the ontology and I could train classifiers for mapping questions to different knowledge areas. Once I reach the leaf nodes, I can give back pre-defined answers. However, my reservation is that mapping chat logs to the ontology would be very tedious. Is there a good way to map the existing QA to the ontology?

Answer 1

This was intended as a comment.

I think this question is not receiving enough attention because it is too difficult to answer without trying several approaches first.

Your third idea is a nice one, but LSA is most likely not going to be able to help you choose clusters with the necessary granularity. For example, you may get a cluster of "broken things" but not to the specificity that is required to be helpful to the customer. A similar approach would be using Hierarchical Dirichlet Processes, but again, you may not get the clusters that you want.

I wouldn't be so eager to try your fourth option because it requires a lot of work and I'm not sure you're going to get great results unless your customer support deals with very well-defined questions and in that case, you're going to have to build your own ontology from scratch.

In your situation, I would be curious if I can get information about the inner workings of Watson (obviously, the purpose is only to have a general idea of some steps that are sensible enough to try.) Here is a descriptive image that could be helpful from Wikipedia:

The main idea is to segment questions into its components (using a regular chunking), then come up with several possibilities (hypothesis generation) of what the question might refer to (maybe this step can be implemented as a clustering). The third stage is rating the previous hypothesis based on available information. Finally, you choose the more likely hypothesis. There is a feedback involved in this process using sample questions and their answers. It would be interesting to see if the rating stage can be modeled in terms of a Bayesian or an incremental approach to allow for a feedback.