How to collect info about unseen bugs given user's comments/feedbacks? [closed]

Question

I have a dataframe which looks like:

user_id, comment
0, 'Functional but Horrible UI'
1, 'Great everything works well'
2, 'I struggled finding plus button because of theme colors in dark mode'
3, 'Keeps stopping on Android 10'
4, 'I like the functionaity but color theme could be better'
5, 'Consistently crashing. Uninstalled'
6, 'Good overall'
7, 'sfdfsdlfksd'
8, 'I lost in complex settings'
9, 'Configuring app is really a headache'
10, 'aaaaaaaaaaaaa'

And I want to figure out some data science approach to pluck out information about what users are struggling with and which issues appeared how much and stuff like this. Even some simple output would be good for me so that we know which parts of app to focus on more. Like for sample above I am aiming for an output as simple as:

problems = {
'color_theme': 3,
'app_settings': 2,
'crashing' : 2}

So I kinda wants labeling and how much time a label is occured based on to which label a review belongs. But the problem is I cannot train a model with predefined labels because:

1. I do not have labels for reviews. If we have to go through each review to know what problem is it talking about (i.e. to label it), we would just have filed it as well and would know what we have to work on.

2. I do not know in advance what problems are gonna come in future so even if we somehow label all at some point in time, it wouldn't be enough as some unseen problem may come and we have to do again.

3. Even if we have a system of labeling somehow, how would we update model, like do we define a new model with a different architecture for ever changing labels?

So under these circumstances, I was trying to figure out an AI approach to ease in my situation. I am pretty good at python and do have working knowledge of keras/tensorflow and other libraries but none of them seem to have such flexible model approach. I was going through Google Cloud Platform's AI platform as well but it could do sentiment analysis to an extent but not understand in an app context that e.g. button is a part of UI and color as well. So how could I approach this problem in a more elegant way?

Answer 1

It sounds like you are looking for an unsupervised learning approach (meaning you don't need to manually label your data).

Something like k-means clustering could work well. This would allow you to group you comments into k distinct clusters. You could then view counts of comments in those clusters and explore the clusters to determine their meaning.

In order to perform the clustering, you need to transform your data from text to a numerical vector space. A common approach would be tf-idf, but you may find that something else works better.

Since you mentioned Python, both k-means and tf-idf can be accomplished using sklearn:

• sklearn.cluster.KMeans
• sklearn.feature_extraction.text.TfidfVectorizer

There's a pretty nice example of k-means clustering using tf-idf on Kaggle.

Answer 2

You can use any standard English language embedding. You could then classify it based on sentiment and retain the ones with negative sentiment. You would then need to extract the sentiment and also do a topic modelling to extract the topic.

'Configuring app is really a headache' could then be broken down to:

• Negative Sentiment
• Words carrying sentiment - headache
• Topic - App configuring

You could then try clustering based on topics and come out with the top recommendations. There are good topic modelling libraries and Attention could be used to highlight words carrying sentiment.

All of this will be completely unsupervised and can be run in online mode.

Also 'labelling' the data can be another approach. The 3 problems to labelling that you have highlighted can be overcome. You could label an initial set as a 1-time activity. If your labelled data is divergent enough, it should elegantly classify or segregate subsequent issues into one of the labels. In case none of the labels fit strongly then that data can be marked for manual triaging. You would pretty quickly cover most of the scenarios