# Find words related to high or low score

I am working on text analysis problem.

Person X can log in his goals and his actions to achieve his goal. Also their score is calculated based on some formula to measure progress of the goal.

For ex:

• Goal1 : Loose weight ... Score: 70%
• Goal2: Finish reading this book.... Score: 10%
• Action1: Go to the gym
• Action2: Drink water
• Action3: Read one page daily

Goals and actions don't have any relationship in my DB, so I joined them based on the person.

## I have as an input

+--------------------------+---------------------+--------+-------+
|           Goal           |       Action        | Person | Score |
+--------------------------+---------------------+--------+-------+
| Lose weight              | Go to the gym       | X      |    70 |
| Lose weight              | Drink water         | X      |    70 |
| Lose weight              | Read one page daily | X      |    70 |
| Finish reading this book | Go to the gym       | X      |    10 |
| Finish reading this book | Drink water         | X      |    10 |
| Finish reading this book | Read one page daily | X      |    10 |
+--------------------------+---------------------+--------+-------+


I want to detect what are the good actions that can have high score and what are the bad actions that can have low score.

## My Approach:

1. Apply clustering algorithm on Goals to find similar groups of goals then study their actions.
2. My Text data dont have the type of action in it, it usually starts with a noun, so I manually created a new column for Actions like below, to make it appear in action form, also to unify the meaning of other actions into one.

+----------------------------------------+--------------------------+
|              Action Title              | Actions (Manually filled)|
+----------------------------------------+--------------------------+
| Sending Video Via whatsapp             | send video               |
| WhatsApp the video                     | send video               |
| WhatsApp Video Broadcasting for Higher | send video               |
| Sending Video Via Whatsapp             | send video               |
| Send video                             | send video               |
| Video Broadcast                        | send video               |
| Send the video                         | send video               |
| Sending Video via Whatsapp             | send video               |
| Share video with 10 contact customers  | send video               |
| New Strategy Owners Contacts           | create strategy contacts |
| New strategy unit contact              | create strategy contacts |
+----------------------------------------+--------------------------+

3. I used R to build a term document matrix using bigram tokenizer (combination of 2 words).

4. I found words appearing only in high score (0-75).
5. I found words appearing only in low score (0-75).
6. I calculated probability of common words to know if it will occur more in high or low.

Conclusion : I concluded step 4 are result of good actions while step 5 are results of bad actions. The common words are either added to step 4 or 5 depending on probability.

## Suggestion

• Am I going in the right path?
• Is there a ready API or a way to automate step 2.

I did step 2 because when I count the words appearing in either high or low, I will find big number. Otherwise, depending only on bigrams will give me a word frequency of 1 for each word.

• Do you have a fixed set of actions? or they are written freely by users as plain text? – Abdulrahman Bres Apr 11 '18 at 4:19
• @Abdo they are written freely by users – sara Apr 11 '18 at 6:28